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IMAGE AND VOICE 

corded image signal multiplex data or an arbitrary com- 
bination thereof, are stored in storage means. In this re- 
production information data, the user can describe and 
edit by freely and easily combining the image data, au- 
dio data, and image/audio multiplex data. The CPU 
reads out image data and audio data from the storage 
means according to the reproduction information data, 
and decodes them, and converts the data format if nec- 
essary, and reproduces the image and audio in synchro- 
nism by using synchronization control means. 
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Description 
TECHNICAL FIELD 

[0001] The present invention relates to a image and 
audio reproducing apparatus and method for reproduc- 
ing image signals and audio signals acquired by a image 
iHKing device such as a digital still camera or a digital 
video camera, automatically and continuously. 

BACKGROUND ART 

(0002] An example of video display system using a 
digital camera is disclosed in Japanese Laid-open Pat- 
en No 11-88808. Herein, as shown in Fig. 37, a file 
rv-.mc ol a image file 3701 selected by the user on a 
diqiul camera is created as a displaylrTanagement in- 
lo'mation file 3702 on a memory card (flash memory 
cnrd) A display control unit 3705 for controlling a display 
unit 3703 such as a liquid crystal display device in Fig. 
37 obtains file names of images to be displayed from 
the display management information file 3702 of the 
memory card by a display management information ac- 
quiring unit 3704. and sequentially displays on the dis- 
play unit 3703. 

[0003] In this prior art. however files being handled 
are only the files recording the image signals, and it is 
not assumed, for example, to reproduce independent 
audio data or audio data recorded simultaneously with 
taking images, in synchronism with images, and such 
technical means is not proposed 

SUMMARY OF THE INVENTION 

[0004] The invention addresses the problems dis- 
cussed above, and it is hence an object thereof to 
present a apparatus and method for reproducing image 
data, audio data, and image and audio multiplexed data 
continuously, as being freely and easily selected and 
combined by the user. 

[0005] To deal with the problems . the image and audio 
reproducing apparatus and method of the invention is 
for storing at least one of recorded image signal data, 
recorded audio signal data, and image/audio signal mul- 
tiplex data, and reproduction information data describ- 
ing the information about reproduction of at least one of 
the recorded image signal data, the recorded audio sig- 
nal data, and the recorded image/audio signal multiplex 
data or an arbitrary combination thereof, independently 
in at least one storage means individually. 
[0006] In this apparatus or method, and in various 
modified forms shown below, according to the informa- 
tion about reproduction, the user can freely and easily 
combine data of various formats, such as a data file re- 
cording image signals, a data file recording audio sig- 
nals, and a data file multiplex recording image signals 
and audio signals, and can reproduce easily and con- 
tinuously, in synchronism between the image signals 



and the audio signals, by adding various information to 
image and audio, changing the sequence of reproduc- 
tion, or making other changes. 
[0007] In the invention, the reproduction information 
5 data may include the information for designating the da- 
ta to be reproduced among the data stored in the stor- 
age means. 

[0008] Also, the reproduction information data may in- 
clude the information showing the sequence of repro- 

10 ducing the data stored in the storage means. 

[0009] Also, the reproduction information data may in- 
clude the information showing the duration of reproduc- 
ing the data stored in the storage means. 
[0010] Also, the reproduction information data may in- 

15 elude the information showing the time of reproducing 
the data stored in the storage means. 
[0011] In the reproduction information data, when 
designating the recorded image/audio signal multi- 
plexed data, the image signal portion and audio signal 

20 portion of the recorded image/audio signal multiplexed 
data can be designated separately as the object of re- 
production. 

[0012] Also in the reproduction information data, the 
information about reproduction of the data is recorded 
25 at least in one group each, and one group includes at 
least one piece of information for designating the data 
to be reproduced. 

[0013] Also in the reproduction information data, the 
information about reproduction of the data is recorded 

30 at least in one group each, and one group includes one 
piece each of the information for designating either the 
image signal portion of the recorded image signal data 
orthe recorded image/audio signal multiplex data, orthe 
audio signal portion of the recorded audio signal data or 

35 the recorded image/audio signal multiplex data. 

[0014] Further in the group of reproduction informa- 
tion data, when writing the information for designating 
the recorded image/audio signal multiplex data, the in- 
formation for designating the recorded image signal da- 

-to ta, the image signal portion of the recorded image/audio 
signal multiplex data, the recorded audio signal data, 
and the audio signal portion of the recorded image/audio 
signal multiplex data is not written in the same group. 
[0015] In the invention and various modified exam- 

15 pies, new reproduction information data can be created 
by editing reproduction information data in the group 
unit. 

[0016] Also in the invention and various modified ex- 
amples, the data format of recorded image signal data, 
50 recorded audio signal data, or recorded image/audio 
signal multiplex data reproduced according to the repro- 
duction information data can be converted by format 
converting means. 

[0017] Also in the invention and various modified ex- 
55 amples, the image signals and audio signals repro- 
duced according to the reproduction information data 
can be coded by encoding means. 
[0018] Also in the invention and various modified ex- 
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amples, combined information data storing the informa- 
tion combining at least one of data the recorded image 
signal recorded audio signal data, and recorded image/ 
audio signal multiplex data is stored in the storage 
means, and the reproduction information data may in- 5 
elude the information for designating the data to be re- 
produced, and the information for designating at least 
one combined information data. 
[0019] Also in the invention and part of various mod- 
ified examples, the information for designating the data io 
to be reproduced may be a name given to the data to 
be reproduced. 

[0020] Also in the invention and various modified ex- 
amples, the image signals and audio signals are repro- 
duced from the data held in the storage means accord- « 
ing to the reproduction information data by using repro- 
ducing means. 

[0021] In the modified example, the reproducing 
means is at least one of a decoder for compounding da- 
ta, means for reading out data from the storage means, 20 
and a decoder for decoding data. 
[0022] Among the modified examples using com- 
bined information, the combined information file is any 
one of image, audio and text contained in the data file 
recording image signals, data file recording audio sig- 25 
nals, and data file multiplex recording image signals and 
audio signals. 



BRIEF DESCRIPTION OF THE DRAWINGS 



[0023] 



30 



Fig. 1 is a block diagram of an image and audio re- 
producing apparatus in embodiments 1 to 5 of the 

invention. 55 
Fig. 2 is a layout diagram of operation panel of the 
image and audio reproducing apparatus in embod- 
iments 1 to 4. 

Fig. 3 is a schematic diagram of image file, audio 
file, and image/audio multiplex file stored in a stor- *o 
age unit in embodiments 1 to 4. 
Fig. 4 is a diagram showing the content of automatic 
reproduction control file in embodiment 1 . 
Fig. 5 is a flowchart of automatic reproduction op- 
eration in embodiment 1 . 45 
Fig. 6 is an explanatory diagram of image and audio 
reproduced automatically in embodiment 1 . 
Fig. 7 is a diagram showing the content of automatic 
reproduction control file in embodiment 2. 
Fig. 8 is an explanatory diagram of image and audio 50 
reproduced automatically in embodiment 2. 
Fig. 9 is a diagram showing the content of automatic 
reproduction control file in embodiment 3. 
Fig. 10 is a flowchart of automatic reproduction op- 
eration in embodiment 3. 55 
Fig. 11 is an explanatory diagram of image and au- 
dio reproduced automatically in embodiment 3. 
Fig. 12 is a diagram showing the content of auto- 



matic reproduction control file in embodiment 4. 
Fig. 13 is a flowchart of automatic reproduction op- 
eration in embodiment 4. 

Fig. 14 is an explanatory diagram of image and au- 
dio reproduced automatically in embodiment 4. 
Fig. 1 5 is an explanatory diagram of example of writ- 
ing method of file designated by character string 
, *Clv1P_SRC ,, in group in embodiment 4. 
Fig. 16 shows an example of inappropriate writing 
of automatic reproduction control file for explaining 
an example of writing method of file designated by 
character string M CMP_SRC" in group in embodi- 
ment 4. 

Fig. 17 shows an example of appropriate writing of 
automatic reproduction control file for explaining an 
example of writing method of file designated by 
character strinc^CMP_SRC ,, in group in embodi- 
ment 4. 

Fig. 1 8 is a layout diagram of operation panel of the 
image and audio reproducing apparatus in embod- 
iment 5. 

Fig. 19 is a diagram showing the content of auto- 
matic reproduction control file in embodiment 5. 
Fig. 20 is an explanatory diagram of image and au- 
dio reproduced automatically in embodiment 5. 
Fig. 21 is an explanatory diagram of concept of ed- 
iting method of a hew automatic reproduction con- 
trol file from plural automatic reproduction control 
files in embodiment 5. 

Fig. 22 is a block diagram of an image and audio 
reproducing apparatus in embodiment 6. 
Fig. 23 is a block diagram showing other example 
of the image and audio reproducing apparatus in 
embodiment 6. 

Fig. 24 is a block diagram of an image and audio 
reproducing apparatus in embodiment 7. 
Fig. 25 is a block diagram of decoding/combining 
unit in embodiment 7. 

Fig. 26 is a layout diagram of operation panel of the 
image and audio reproducing apparatus in embod- 
iment 7. 

Fig. 27 is a schematic diagram showing an addition- 
al information file stored on the storage unit in em- 
bodiment 7. 

Fig. 28 is a diagram showing the content of auto- 
matic reproduction control file in embodiment 7. 
Fig. 29 is a flowchart of automatic reproduction op- 
eration in embodiment 7. 

Fig. 30 is an explanatory diagram of combination of 
image file, audio file, and additional information file 
reproduced automatically in embodiment 7. 
Fig. 31 is a layout diagram of operation panel of a 
image and audio reproducing apparatus in embod- 
iment 8. 

Fig. 32 is a diagram showing the content of auto- 
matic reproduction control file in embodiment 8. 
Fig. 33 is a flowchart of automatic reproduction op- 
eration in embodiment 8. 
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Fig. 34 is an explanatory diagram of reproduction 
duration of image and audio reproduced automati- 
cally in embodiment 8. 

Fig. 35 is a diagram showing another example of 
automatic reproduction control file in embodiment 
8. 

Fig. 36 is a diagram showing another example of 
automatic reproduction control file in embodiment 

8. 

Fig. 37 is a block diagram of an automatic reproduc- 
ing apparatus in a prior art. 

DETAILED DESCRIPTION OF THE PREFERED 
EMBODIMENTS 

[0024] Referring now to the drawings, preferred em- 
bodiments of the invention are described in detail below. 

(Embodiment 1) 

[0025] Fig. 1 is a block diagram of a image and audio 
reproducing apparatus in embodiments 1 to 5 of the in- 
vention. In Fig. 1 , a storage unit 101 stores image and 
audio digital data acquired (coded) by a device such as 
a digital still camera, digital video camera, or scanner. 
In all other embodiments described below, the storage 
unit 101 may be either fixed means such as a hard disk 
device, or detachable means such as a flash memory 
card. Storage medium may be a semiconductor memo- 
ry, magnetic disk, magnetic tape, optical disk, or the like. 
A image file 1 02 is a file of image data recorded in the 
storage unit 101 , an audio file 103 is similarly a file of 
audio data, and a image/audio multiplex file 104 is sim- 
ilarly a file in which image data and audio data are mul- 
tiplexed and stored in the same file. An automatic repro- 
duction control file 1 05 is a file storing information show- 
ing reproduction data and reproduction sequence when 
reproducing the image file 102, audio file 103, and im- 
age/audio multiplex file 104 continuously in a combina- 
tion. In Fig. 1 , however, for the sake of simplicity, the 
image file 1 02, audio file 1 03, image/audio multiplex file 
104, and automatic reproduction control file 105 are 
shown by one piece each, but not limited to this exam- 
ple, actually, each file may be present in a plurality. 
Meanwhile, all of the image file 102, audio file 103, and 
image/audio multiplex file 1 04 are not always recorded 
in the storage unit 101 , but, for example, the image file 
1 02 may not be recorded in the storage unit 101. 
[0026] A reading unit 106 is means for reading out 
files recorded in the storage unit 101 . and it is one of 
reproducing means. A writing unit 1 07 is means for writ- 
ing an automatic reproduction control file 105 into the 
storage unit 101 . A decoder 108 is one of reproducing 
means, and is means for decoding and reproducing the 
image file 1 02, audio file 1 03, and image/audio multiplex 
file 104 being read out from the storage unit 101. The 
reproducing means may refer to the decoder 108 only, 
or both the decoder 108 and reading unit 106, depend- 



ing on the case. At this time, synchronization control unit 
109 controls decoding by the decoder 108 so that the 
image signal and audio signal issued from the decoder 
108 according to the information of the automatic repro- 

5 duction control file 1 05 may be in the specified synchro- 
nzed relation exactly conforming to the information of 
the automatic reproduction control file 105. 
[0027] A central processing u n it (C PU) 1 1 0 generates 
an automatic reproduction control file 105 depending on 

10 the input from an operation key 111, controls the reading 
unit 106 or synchronizing control unit 109 on the basis 
of the information of the automatic reproduction control 
file 105, and controls reading of image file, decoding and 
reproduction of the file being read out in the decoder 

15 108. 

[0028] A display unit 112 is a display device such as 
a liquid crystal display device orcatnode- ray tube (CRT) 
for displaying the image signal decoded by the decoder 
1 08, and a speaker 1 1 3 is for reproducing the audio sig- 
20 nal decoded by the decoder 1 08. 

[0029] Fig. 2 is a layout diagram of operation panel of 
the image and audio reproducing apparatus in embod- 
iments 1 to 4. 

[0030] Keys 201 to 209 are keys for composing the 

25 operation key 1 1 1 shown in Fig. 1 , and a key input signal 
is sent to the CPU 110 shown in Fig. 1. Principal func- 
tions of each key are as follows. 
[0031] When a image file reproduction mode select 
key 201 is pressed, it results in reproduction mode of 

30 image file on the storage unit 101, and the CPU 110 con- 
trols the reading unit 106. reads out one of the image 
file and image/audio multiplex file recorded in the stor- 
age unit 101 , and decodes by the decoder 108, so that 
the image of the file being read out is displayed in the 

35 display unit 112. 

[0032] When an audio file reproduction mode select 
key 202 is pressed, it results in reproduction mode cf 
audio file on the storage unit 1 01 , and the CPU 110 con- 
trols the reading unit 1 06, reads out one of the audio file 

40 and image/audio multiplex file recorded in the storage 
unit 101 , and decodes by the decoder 108, so that the 
audio of the file being read out is reproduced from the 
speaker 113. 

[0033] When a image/audio multiplex file reproduc- 
es tion mode select key 203 is pressed, it results in repro- 
duction mode of image/audio multiplex file on the stor- 
age unil 101, and the CPU 1 1 0 controls the reading unit 
106, reads out one of the image/audio multiplex files re- 
corded in the storage unit 101 , and decodes by the de- 
50 coder 108. so that the image of the file being read out 
is displayed in the display unit 112, while the sound is 
reproduced from the speaker 113. 
[0034] A send key 204 is a key to be pressed while 
reproducing other file than the file being presently repro- 
55 duced, and a return key 205 is a key for reproducing the 
file once reproduced immediately before. A decision key 
206 is a key pressed when selecting the reproduced file 
as the object of automatic reproduction. 
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[0035] When a group key 207 is pressed, the CPU 110 
provides a specific file group with a group ID number, 
and a file name is written in an automatic reproduction 
control file together with a specific character string. 
[0036] When a new reproduction control file create 
key 208 is press d, the CPU 110 controls the writing unit 
107, and newly creates an automatic reproduction con- 
trol file on the storage unit 101 . 
[0037] When an automatic reproduction key 209 is 
pressed, the CPU 110 controls the reading unit 106 and 
synchronization control unit 109 on the basis of the in- 
formation of the automatic reproduction control file 1 05, 
and automatically reads out and reproduces the file on 
the storage unit 101. 

[0038] Fig, 3 is a schematic diagram of image file, au- 
dio file, and image/audio multiplex file stored in storage 
unit 101 in embodiments 1 to 4. In Fig. 3, image files A, 
B, C are files in which image data are stored. Audio files 
A, B, C are files in which audio data are slored. Image/ 
audio multiplex files A, B are files in which image and 
audio data are multiplexed and stored in one file, in the 
following explanation, it is supposed that these eight 
files shown in Fig. 3 are recorded on the storage unit 
101. 

[0039] Concerning the embodiment having such con- 
figuration, the operation is explained below. 
[0040] When the user reproduces the image file, au- 
dio file, and image/audio multiplex file recorded on the 
storage unit 101 by combining freely, first, an automatic 
reproduction control file 105 is created in the following 
procedure. 

[0041] First, of the eight files shown in Fig. 3, if the 
user desires to reproduce the image portions of the im- 
age file B and image/audio multiplex file A continuously 
in this sequence as for the image, to reproduce the audio 
portions of the audio file A and image/audio multiplex 
file B continuously in this sequence as for the audio, and 
further to reproduce the audio file A synchronized with 
start of reproduction of image file B, the operation is ex- 
plained by referring to the content of the automatic re- 
production control file in Fig. 4. 
[0042] To begin with, the user presses the new repro- 
duction control file create key 208. Receiving this key 
input, editing of automatic reproduction control file is 
newly started, and the CPU 1 10 controls the writing unit 
107, and creates a file in a file name of, for example, 
"AUTOPLYV on the storage unit 101. 
[0043] Next, the user presses the image file reproduc- 
tion mode select key 201 in order to select the image to 
be reproduced automatically, and reproduces and dis- 
plays the image file on the storage unit 1 01 in the display 
unit 112. At this timo : for example, imago file A is repro- 
duced in the display unit 112, and by changing the dis- 
play image by using the send key 204 or return key 205, 
the image file B, image file C, image portion of image/ 
audio multiplex file A, and image portion of image/audio 
multiplex file B are supposed to be displayed sequen- 
tially. When the user desires to reproduce the image file 



B and image portion of image/audio multiplex file A con- 
tinuously in this sequence, the user manipulates the 
send key 204 or return key 205. and displays the image 
file B to be reproduced in the first place in the display 
5 unit 112. Then the decision key 206 is pressed. The CPU 
1 1 0 has a temporary memory for storing the information 
temporarily in its inside, and when the decision key 206 
is pressed, the file name of the image file B is recorded 
in this temporary memory in succession to the character 
w string H IMG__SRC= n for indicating that the user has se- 
lected this file as the image to be reproduced in the re- 
production mode of the image file. Consequently, when 
the send key 204 or return key 205 is pressed, the image 
of the image/audio multiplex file A is displayed. The de- 
15 cision key 206 is pressed again. As a result, the file 
name of the audio/image multiplex file A is recorded in 
this temporary memory of the CPU 110 in succession to 
the character string *'IMG_SRC=" similarly. In this pro- 
cedure, the image file B and image/audio multiplex file 
20 a are selected, and the file names are recorded in the 
CPU 110 together with the character string 
,, IMG_SRC=: ,, in the desired sequence of reproduction. 
[0044] Next, to select the audio to be reproduced au- 
tomatically, the audio file reproduction mode select key 
25 202 is pressed, and the audio vile on the storage unit 
101 is reproduced from the speaker 113. At this time, in 
the speaker 113, for example, when the audio file A is 
reproduced, and by changing the audio to be repro- 
duced by the send key 204 or return key 205, it is sup- 
30 posed that the audio file B, audio file C, audio portion of 
image/audio multiplex file A, and audio portion of image/ 
audio multiplex file B are reproduced sequentially from 
the speaker 113. When the user desires to reproduce 
the audio file A and audio portion of image/audio multi- 
35 plex file B continuously in this sequence, the user ma- 
nipulates the send key 204 or return key 205, and re- 
produces the audio file A to be reproduced in the first 
place from the speaker 113. Then the decision key 206 
is pressed. In the temporary memory of the CPU 110, 
40 the file name of the audio file A is recorded in succession 
to the character string M SND_SRC=" for indicating that 
the user has selected this file as the audio to be repro- 
duced in the reproduction mode of the audio file. Con- 
sequently, when the send key 204 or return key 205 is 
45 pressed, the audio portion of the image/audio multiplex 
file B is reproduced. The decision key 206 is pressed 
again. As a result, the file name of the image/audio mul- 
tiplex file B is recorded in the temporary memory of the 
CPU 110 in succession to the character string 
so "SND_SRC^" similarly. In this procedure, the audio file 
A and image/audio multiplex file B are selected, and the 
file names arc recorded in the CPU 110 together with 
the character string W SND_SRC=" in the desired se- 
quence of reproduction. 
55 [0045] In this state, consequently, when the group key 
207 is pressed, the CPU 110 recognizes these selected 
image file B and image/audio multiplex file A, and audio 
file A and image/audio multiplex file B as one group, and 
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provides with an original ID number (for example 001), 
and controls the writing unit 107, and write the group 
original ID number and the content stored in the tempo- 
rary memory of the CPU 1 1 0 in the automatic reproduc- 
tion control file "AUTOPLY1 " on the storage unit 101. 
Fig. 4 shows the content of automatic reproduction con- 
trol file " AUTO PLY 1" written in this manner. In Fig. 4, 
"GID" is group ID number, and it is 001 in the example 
in Fig. 4. The character string W IMG_SRC M shows the 
image file or image portion of image/audio multiplex file, 
and it shows that the image file B and image portion of 
image/audio multiplex file A are the images selected by 
the user to be reproduced. The character string 
"SND_SRC" shows the audio file or audio portion of im- 
age/audio multiplex file, and it shows that the audio file 
A and audio portion of image/audio multiplex file B are 
the audios selected by the user to be reproduced. In par- 
ticular, concerning the image/audio multiplex file, the im- 
age portion and audio portion can be separately desig- 
nated by the character strings M IMG_SRC" and 
"SND_SRC M , so that the image/audio multiplex file can 
be handled as a image file composed of image data only 
or an audio file composed of audio data only. 
[0046] Of these files indicated by "IMG_SRC" and 
"SND_SRC", the file of the smaller number of lines is 
the file to be reproduced first in time. 
[0047] This is the procedure of creating the automatic 
reproduction control file 105. Next, reproduction of file 
on the basis of the automatic reproduction control file 
105 shown in Fig. 4 is explained. 
[0048] When the user presses the automatic repro- 
duction key 209, the CPU 110 controls the reading unit 
106, reads out the automatic reproduction control file 
105, and acquires its content. The CPU 110 searches 
for the GID from the first line of the automatic reproduc- 
tion control file 105, and recognizes from the first GID 
to just before next GID as one group. The final group in 
the automatic reproduction control file 105 is from the 
final GID to the final line of the file. In this embodiment, 
there is only one group, that is, one group is formed from 
the first GID to the final line of the automatic reproduc- 
tion control file 105. 

[0049] The automatic reproduction control file 1 05 in- 
cludes file names in the sequence of reproduction. The 
image file or the image portion of image/audio multiplex 
file is designated by character string "IMG.SRC", and 
the audio file or the audio portion of image/audio multi- 
plex file is designated by character string *'SND_SRC". 
Accordingly, the CPU 1 1 0 judges whether the object file 
relates to image reproduction or to audio reproduction 
by checking the character string. As a result, as for the 
image, the imago file B and the image portion of image/ 
audio multiplex file A are read out in this sequence from 
the storage unit 1 01 , and as for the sound, the audio file 
A and the audio portion of image/audio multiplex file B 
are read out . and decoded and reproduced. At this time, 
from the description of the first line of the automatic re- 
production control file 105, these files are judged to be 



one group identified with the same group ID, and the 
synchronization control unit 1 09 control decoding of the 
decoder 108 so as to synchronize start of reproduction 
of image file B to be reproduced first among the image 
5 files included in the group and start of reproduction of 
audio file A to be reproduced first among the audio files 
included in the same group. 

[0050] Specifically, the CPU 110 sends the informa- 
tion showing the image and audio to be reproduced syn- 

10 chronously to the synchronization control unit 1 09. The 
decoder 108 has a temporary memory for storing the 
decoded result provisionally within itself, and decodes 
the file being read out from the storage unit 101, and 
once writes into the temporary memory sequentially. 

*5 This is the means for compensating for time error if dif- 
ference occurs in the time of decoding depending on the 
files. The synchronization control unit 109 monitors the 
progress of decoding of picture and sound to be repro- 
duced synchronously, and when decoded completely 

20 until it is ready to reproduce the picture and sound si- 
multaneously: the picture and sound stored in the tem- 
porary memory are reproduced at the same time. Thus, 
the image reproduction and audio reproduction are syn- 
chronized. 

25 [0051] This series of flow of reproduction of files is 
shown in a flowchart in Fig. 5. In Fig. 5, step 501 is to 
wait for input from the automatic reproduction key 209, 
and when the automatic reproduction key 209 is 
pressed, the process goes to step 502. At step 502, the 

30 CPU 110 controls the reading unit 106, and reads out 
the automatic reproduction control file 1 05 from the stor- 
age unit 101. At step 503, the CPU 110 searches for 
group ID number from the automatic reproduction con- 
trol file 105 being read out, and detects the group. Fur- 

35 ther, at step 504, in every detected group, the file name 
designated by character string "IMG^SRC" indicating 
the image is detected, and at step 505, similarly, the file 
name designated by character string "SND_SRC M indi- 
cating the sound is detected. Step 506, as explained 

^o above, is to control reading of file from the storage unit 
101, decoding by the decoder 108, and synchronized 
reproduction of picture and sound by the synchroniza- 
tion control unit 109. 

[0052] Such reproduction of image and audio is sche- 
45 matically shown in Fig. 6. That is, the picture is repro- 
duced in the sequence of image file B and image portion 
of image/audio multiplex file A, and synchronously with 
start of reproduction of image file B, audio file A is re- 
produced and then audio portion of image/audio multi- 
50 piex file B is reproduced. Thus, the image and audio can 
be reproduced as designated by the user. 
[0053] According to the embodiment, the image file 
and audio file to be reproduced synchronously are spec- 
ified in a group form in the automatic reproduction con- 
55 trol file 105 as shown in Fig. 4. According to this auto- 
matic reproduct:on control file 105, reading of file from 
the storage unit 101 and decoding by the decoder 108 
are controlled so as to synchronize start of reproduction 
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of image file to be reproduced first among the image files 
included in the group and start of reproduction of audio 
file to be reproduced first among the audio files included 
in the same group. Therefore, the image data, audio da- 
ta, and image and audio multiplexed data can be freely 
and easily combined by the user, and the desired image 
and audio can be reproduced synchronously. 

(Embodiment 2) 

[0054] In embodiment 1 of the invention, a group con- 
sisting of plural files is described in the automatic repro- 
duction control file 105, and in this case, the image and 
audio can be synchronized only at the start point of re- 
production of the files included in the group, and next 
image and audio may not be always synchronized. A 
further synchronized reproduction in detail of image and 
audio may be considered. An embodiment assuming 
such case is explained by referring to Fig. 7 showing the 
content of automatic reproduction control file. The con- 
figuration of the image and audio reproducing apparatus 
of the embodiment is the same as shown in the block 
diagram in Fig. 1 , and the layout of the operation panel 
of the image and audio reproducing apparatus is the 
same as shown in Fig. 2, and the explanation is omitted. 
[0055] For example, among the files shown in Fig. 3, 
image file A and image file B are reproduced in this se- 
quence, and start of reproduction of image file A and 
audio file B is synchronized, and further start of repro- 
duction of image file B and audio portion of image/audio 
multiplex file B is synchronized, and in this case, first, 
the automatic reproduction control file 105 is created in 
the following procedure. 

[0056] To begin with, the user presses the new repro- 
duction control file create key 208 in the same way as 
in the case of embodiment 1 . As a result, a file in a file 
name of, for example, "AUTOPLY2 M is created on the 
storage unit 101. 

[0057] Next, the user presses the image file reproduc- 
tion mode select key 201 for selecting the image to be 
reproduced automatically, and the image file on the stor- 
age unit 101 is reproduced and displayed in the display 
unit 112 : and the display image is changed by the send 
key 204 or return key 205 : and the image file A is dis- 
played. Then the decision key 206 is pressed. As a re- 
sult, in succession to the character string "IMG_SRC= M , 
the file name of the image file A is recorded in the tem- 
porary memory of the CPU 110. Further, to select the 
audio to be reproduced automatically, the audio file re- 
production mode select key 202 is pressed, and the au- 
dio file on the storage unit 101 is reproduced from the 
speaker 113, and the reproduction audio is changed by 
the send key 204 or return key 205, and the audio file B 
is reproduced. Then the decision key 206 is pressed. As 
a result, in succession to the character string 
M SND_SRC='\ the file name of the audio file B is record- 
ed in the temporary memory of the CPU 110. 
[0058] In this state, when the group key 207 is 



pressed, the CPU 110 recognizes these selected image 
file A and audio file B as one group, and provides with 
an original ID number (for example 001). and controls 
the writing unit 107, and writes the group original ID 

5 number and the content stored in the temporary memory 
of the CPU 1 1 0 in the automatic reproduction control file 
"AUTOPLY2" on the storage unit 101 . 
[0059] In the same way as mentioned above, the im- 
age file B and audio portion of image/audio multiplex file 

10 B are reproduced and selected, and the group key 207 
is pressed. As a result, the CPU 1 1 0 recognizes the im- 
age file B and audio portion of image/audio multiplex file 
B as one group, and provides with an original ID number 
(for example 002). It further controls the writing unit 1 07, 

15 and writes the group original ID number and the content 
stored in the temporary memory of the CPU 110 in the 
automatic reproduction control file "AUTOPLY2" on the 
storage unit 101. Fig. 7 shows the content of the created 
automatic reproduction control file "AUTOPLY2". In Fig. 

20 7, "GID" is group ID number, and in the example in Fig. 
7, ID numbers are given to two groups. In the same man- 
ner as in Fig. 4, the character string "IMG_SRC" shows 
the image file or image portion of image/audio multiplex 
file, and the character string "SND_SRC" shows the au- 

25 dio file or audio portion of image/audio multiplex file. Of 
these file groups, the file group having the GID of the 
smaller number of lines is the file groupto be reproduced 
first in time. 

[0060] This is the procedure of creating the automatic 

30 reproduction control file 105. Next, reproduction of file 
on the basis of the automatic reproduction control file 
105 shown in Fig. 7 is explained. 
[0061] When the user presses the automatic repro- 
duction key 209, the CPU 110 controls the reading unit 

35 106, reads out the automatic reproduction control file 
105, and acquires its content. The CPU 110 searches 
for the GID from the first line of the automatic reproduc- 
tion control file 105, and recognizes from the first GID 
to just before next GID as one group. The final group in 

40 the automatic reproduction control file 105 is from the 
final GID to the final line of the file. From the description 
of the automatic reproduction control file 1 05, image file 
A and audio file B are recognized as one group, and 
image file B and audio portion of image/audio multiplex 

45 file B as one group. The image is decoded and repro- 
duced by reading out the image file A and image file B 
in this sequence from the storage unit 101, and the audio 
is similarly decoded and reproduced by reading out the 
audio file B and audio portion of image/audio multiplex 

so file B in this sequence from the storage unit 1 01 . At this 
time, since the image file A and audio file B form one 
group, and the image file B and audio portion of image/ 
audio multiplex file B form one group, decoding by the 
decoder 1 08 is controlled by the synchronization control 

55 unit 109 so as to synchronize start of reproduction of 
image file A and start of reproduction of audio file B, and 
start of reproduction of image file B and start of repro- 
duction of audio portion of image/audio multiplex file B. 
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The flowchart of this series of reproduction of files is the 
same as in embodiment 1 of the invention shown in Fig. 
5. Such reproduction of image and audio is schemati- 
cally shown in Fig. 8, and the user can reproduce the 
image and audio freely as designated. 
[0062] Thus, according to embodiment 2, a plurality 
of groups consisting of plural files each are placed in 
one automatic reproduction control file, and reproduc- 
tion of image and audio can be synchronized in detail, 
and therefore the image data, audio data, and image 
and audio multiplexed data can be freely and easily 
combined by the user, and the image and audio can be 
reproduced synchronously. 

(Embodiment 3) 

[0063] In embodiments 1 and 2 of the invention, plural 
files in the automatic reproduction control file 105 are 
provided with a group ID number, and plural files are 
regarded as one group, and the image and audio are 
reproduced synchronously. In these examples, the im- 
age/audio multiplex file is divided into image portion and 
audio portion, and recorded in the automatic reproduc- 
tion control file 105 in such divided state. However, the 
image/audio multiplex file taken by a digital still camera 
or digital image camera is a file recording image and 
audio at the same time, and therefore the image and 
audio are recorded synchronously in this file. When re- 
producing such file, if it is recorded in the automatic re- 
production control file by once dividing into image por- 
tion and audio portion, it is not only complicated for the 
user, but also it may cause confusion. 
[0064] Accordingly, in this embodiment, the image/ 
audio multiplex file is reproduced from the automatic re- 
production control file while maintaining the synchro- 
nized relation of image and audio without separating into 
image portion and audio portion, and this is explained 
by referring to Fig. 9 showing the content of automatic 
reproduction control file. The configuration of the image 
and audio reproducing apparatus of the embodiment is 
the same as shown in the block diagram in Fig. 1 , and 
the layout of the operation panel of the image and audio 
reproducing apparatus is the same as shown in Fig. 2, 
and the explanation is omitted. 
[0065] For example, among the files shown in Fig. 3, 
If the user desires to reproduce image/audio multiplex 
file A and image/audio multiplex file B continuously in 
this sequence, the automatic reproduction control file is 
created in the following procedure. 
[0066] To begin with : the user presses the new repro- 
duction control file create key 208 in the same way as 
in the case of embodiment 1 . As a result, a file in a file 
name of ; for example, "AUTOPLY3" is created on the 
storage unit 101 . 

[0067] • Next, the user presses the image/audio multi- 
plex file reproduction mode select key 203 for selecting 
the image/audio multiplex file to be reproduced auto- 
matically, and the image/audio multiplex file on the stor- 



age unit 101 is reproduced. The reproduction file is 
changed by using the send key 204 or return key 205, 
and the image/audio multiplex file A is reproduced, and 
then decision key 206 is pressed. As a result, the file 

5 name of image/audio multiplex file A is recorded in the 
temporary memory of the CPU 11 0, in succession to the 
character string "CMP_SRC=" which indicates that the 
user has selected this file as the image and audio mul- 
tiplexed file to be reproduced in the reproduction mode 

10 of the image/audio multiplex file. Similarly, the image/ 
audk> multiplex file B is reproduced, and the decision 
key 206 is pressed, and then, in succession to the char- 
acter string W CMP_SRC= M , the file name of the image/ 
audio multiplex file B is recorded in the temporary mem- 

'5 oryoftheCPU 110. In this procedure, image/audio mul- 
tiplex file A and image/audio multiplex file B are select- 
ed, and the file names are stored in the CPU 11 0, in the 
desired sequence of reproduction, together with the 
character siring "CMP_SRC=". 

20 [0068] In this state, when the group key 207 is 
pressed, the CPU 1 1 0 recognizes these selected image/ 
audio multiplex file A and image/audio multiplex file B 
as one group, and provides with an original ID number 
(for example 001). Further it controls the writing unit 107, 

25 and writes the group original ID number and the content 
stored in the temporary memory of the CPU 110 in the 
automatic reproduction control file "AUTOPLY3" on the 
storage unit 1 01 . Fig. 9 shows the content of the created 
automatic reproduction control file "AUTOPLYS". In Fig. 

30 9, "GID" is group ID number, and "CM P_SRC" is a char- 
acter string showing a image/audio multiplex file, which 
shows that the image/audio multiplex files A and B are 
data selected by the user as the picture and sound to 
be reproduced. Of these file groups, the file group hav- 

35 ing the GID of the smaller number of lines is the file 
group to be reproduced first in time. 
[0069] This is the procedure of creating the automatic 
reproduction control file 105. Next, reproduction of file 
on the basis of the automatic reproduction control file 

40 105 shown in Fig. 9 is explained. 

[0070] When the user presses the automatic repro- 
duction key 209, the CPU 110 controls the reading unit 
106, reads out the automatic reproduction control file 
105, and acquires its content. The automatic reproduc- 
es tion control file 105 includes the file names in the se- 
quence of reproduction. The image/audio multiplex file 
is designaled by the character string M CMP_SRC", and 
it is judged from this character string that the object file 
is a file multiplexing and recording image and audio. As 

so a result, image/audio multiplex file A and image/audio 
multiplex file B are read out from the storage unit 101 
and decoded by the decoder 1 08, and imago and audio 
are reproduced synchronously. 

[0071] Specifically, the CPU 110 sends the informa- 
55 tion showing the files to be reproduced synchronously 
to the synchronization control unit 1 09. The decoder 1 08 
has a temporary memory for storing the decoded result 
provisionally in its inside, and decodes the file being 
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read out from the storage unit 1 01 , and once writes into 
the temporary memory sequentially. This is the means 
for compensating for time error if difference occurs in 
the time of decoding between the image portion and au- 
dio portion in the image/audio multiplex file. The syn- 
chronization control unit 109 monitors the progress of 
decoding of image and audio to be reproduced, and 
when decoded completely until it is ready to reproduce 
the picture and sound simultaneously, the image and 
audio stored in the temporary memory are reproduced 
at the same time. Thus, the image reproduction and au- 
dio reproduction are synchronized. 
[0072] This series of flow of reproduction of files is 
shown in a flowchart in Fig. 10. In Fig. 10, step 501 
through step 503 are the same as in Fig. 5, and are not 
explained herein. Step 100/Us for detecting the file 
name designated by the character string "CMP_SRC tt 
showing the image in every detected group. At step 
1002, the CPU 110 reads out the designated image/au- 
dio multiplex files from the storage unit 1 01 sequentially 
from the earlier group ID number, and controls decoding 
by the decoder 108 and synchronized reproduction of 
image and audio by the synchronization control unit 1 09. 
[0073] Such reproduction of image and audio is sche- 
matically shown in Fig. 11 , and the user can reproduce 
the image and audio as designated. 
[0074] Thus, according to embodiment 3, the image 
and audio multiplexed data is not divided into image por- 
tion and audio portion, and while maintaining the syn- 
chronized state of image and audio, the user can freely 
and easily combine, and reproduce the image and audio 
synchronously. 

(Embodiment 4) 

[0075] In embodiment 3 of the invention, a group con- 
sisting of image/audio multiplex files only is described 
in the automatic reproduction control file 105, and an 
example of automatic reproduction is explained. In this 
embodiment, the user reproduces automatically includ- 
ing image file and audio file, and it is explained by refer- 
ring to Fig. 12 showing the content of the automatic re- 
production control file. The configuration of the image 
and audio reproducing apparatus of the embodiment is 
the same as shown in the block diagram in Fig. 1 , and 
the layout of the operation panel of the image and audio 
reproducing apparatus is the same as shown in Fig. 2, 
and the explanation is omitted. 

[0076] For example, among the files shown in Fig. 3, 
the user desires to reproduce image f ile B ; image/audio 
multiplex file B, and image file C in this sequence, and 
at this time, start of reproduction of image file B and au- 
dio file A is synchronized, the image portion of the im- 
age/audio multiplex file B is synchronized with the audio 
portion of the same file, and further start of reproduction 
of image file C and audio portion of image/audio multi- 
plex file A is synchronized, and in this case, first, the 
automatic reproduction control file is created in the fol- 



lowing procedure. 

[0077] To begin with, the user presses the new repro- 
duction control filecreate key 208 in the same way as in 
the case of embodiment 1 . As a result, a file in a file 
5 name of, for example, "AUTOPLY4" is created on the 
storage unit 101. 

[0078] Next, the user presses the image file reproduc- 
tion mode select key 201 for selecting the image to be 
reproduced automatically, and the image file on the stor- 
10 age unit 1 01 is reproduced and displayed in the display 
unit 112, and the display image is changed by the send 
key 204 or return key 205, and the image file B is dis- 
played. Then the decision key 206 is pressed. As a re- 
sult, the file name of image file B is recorded in the tern- 
is porary memory of the CPU 110, in succession to the 
character string M IMG_SRC=" which indicates that the 
user has selected this file as the image to be reproduced 
in the reproduction mode of the image file. Further, to 
select the audio to be reproduced automatically, the au- 
20 dio file reproduction mode select key 202 is pressed, 
and the audio file on the storage unit 101 is reproduced 
from the speaker 113, and the reproduction audio is 
changed by the send key 204 or return key 205, and the 
audio file A is reproduced. Then the decision key 206 is 
25 pressed. As a result, the file name of the audio file A is 
recorded in the temporary memory of the CPU 110, in 
succession to the character string "SND_SRC=" which 
indicates that the user has selected this file as the audio 
to be reproduced in the reproduction mode of the audio 
30 file. 

[0079] In this state, when the group key 207 is 
pressed, the CPU 1 1 0 recognizes these selected image 
file B and audio file A as one group, and provides with 
an original ID number (for example 001), and controls 
35 the writing unit 107, and writes the group original ID 
number and the content stored in the temporary memory 
of the CPU 110 in the automatic reproduction control file 
"AUTOPLY4" on the storage unit 101 . 
[0080] Next, to select the image/audio multiplex file to 
40 be reproduced automatically, the image/audio multiplex 
file reproduction mode select key 203 is pressed, and 
the image/audio multiplex file on the storage unit 1 01 is 
reproduced. The display image is changed by using the 
send key 204 or return key 205, and the image/audio 
45 multiplex file B is reproduced, and then decision key 206 
is pressed. As a result, the file name of image/audio mul- 
tiplex file B is recorded in the temporary memory of the 
CPU 110, in succession to the character string 
"CIv^SRC^ which indicates that the user has select- 
so ed this file as the image and audio multiplexed file to be 
reproduced in the reproduction mode of the image/audio 
multiplex file. 

[0081] In this state, when the group key 207 is 
pressed, the CPU 110 recognizes the selected image/ 
55 audio multiplex file B as one group, and provides with 
an original ID number (for example 002), and controls 
the writing unit 107, and writes the group original ID 
number and the content stored in the temporary memory 



9 



<EP__ 1199892A1 I > 



17 



EP 1 199 892 A1 



18 



of the CPU 11 0 in the automatic reproduction control file 
"AUTOPLY4" on the storage unit 101 . 
[0082] Again, to select the image to be reproduced 
automatically, the image file reproduction mode select 
key 201 is pressed, and the image file on the storage 5 
unit 101 is reproduced and displayed in the display unit 
1 1 2. and the display image is changed by using the send 
key 204 or return key 205. and the image file C is dis- 
played, and then decision key 206 is pressed. As a re- 
sult, the file name of image file C is recorded in the tern- 10 
porary memory of the CPU 110, in succession to the 
character string "IMG_SRC=:". Further, to select the au- 
dio to be reproduced automatically, the audio file repro- 
duction mode select key 202 is pressed, and the audio 
file on the storage unit 101 is reproduced from the *5 
speaker 113, and the reproduction audio is changed by 
using the send key 204 or return key 205, and the audio 
portion of the image/audio multiplex file A is reproduced, 
Then the decision key 206 is pressed. As a result, the 
file name of image/audio multiplex file A is recorded in 20 
the temporary memory of the CPU 110, in succession 
to the character string "SND_SRC^\ 
[0083] In this state, when the group key 207 is 
pressed, the CPU 110 recognizes the selected image 
file C and audio portion of image/audio multiplex file A 25 
as one group, and provides with an original ID number 
(for example 003), and controls the writing unit 1 07, and 
writes the group original ID number and the content 
stored in the temporary memory of the CPU 11 0 in the 
automatic reproduction control file ,, AUTOPLY4" on the 30 
storage unit 101. Fig. 12 shows the content of the written 
automatic reproduction control file M AUTOPLY4". In Fig. 
12, "GID" is group ID number, three groups are individ- 
ually provided with group ID numbers in the example in 
Fig. 12. In the same manner as in Fig. 4, "IMG.SRC" is 35 
a character string showing a image file or image portion 
of image/audio multiplex file, "SND_SRC" is a character 
string showing an audio file or audio portion of image/ 
audio multiplex file, and, in the same manner as in Fig. 
9, "CMP_SRC" is a character string showing a image/ *o 
audio multiplex file. Of these file groups, the file group 
having the GID of the smaller number of lines is the file 
group to be reproduced first in time, and within a group, 
of the files indicated by character strings M IMG_SRC" 
and others, the file having the smaller number of lines 45 
indicated by the same character string is to be repro- 
duced first. 

[0084] This is the procedure of creating the automatic 
reproduction control file 105. Next, reproduction of file 
on the basis of the automatic reproduction control file so 
105 shown in Fig. 12 is explained. 
[0085] When the user presses the automatic repro- 
duction key 209, the CPU 110 controls the reading unit 
106, reads out the automatic reproduction control file 
105, and acquires its content. The CPU 110 searches 55 
for the GID from the first line of the automatic reproduc- 
tion control file, and recognizes from the first GID to just 
before next GID as one group. The final group in the 



automatic reproduction control file 105 is from the final 
GID to the final line of the file. From the description of 
the automatic reproduction control file 1 05. image file B 
and audio file A are recognized as one group, image/ 
audio multiplex file B as one group, and image file C and 
audio portion of image/audio multiplex file A as one 
group. From the storage unit 101, image file B, audio file 
A, image/audio multiplex file B, image file C, and audio 
portion of image/audio multiplex file A are read out. de- 
coded by the decoder 108. and reproduced. At this time, 
the image file B and audio file A form one group ; image/ 
audio multiplex file B is one group, and image file C and 
audio portion of image/audio multiplex file A form one 
group. Accordingly, as shown in the explanatory dia- 
gram in Fig. 14, decoding by the decoder 108 is control- 
led by the synchronization control unit 109 in the follow 
manner: reproduction of image file B and reproduction 
of audio file A are started synchronously, the image por- 
tion and audio portion of image/audio multiplex file B are 
reproduced synchronously, and finally start of reproduc- 
tion of image file C and start of reproduction of audio 
portion of image/audio multiplex file A are synchronized. 
This series of flow of reproduction of files is shown in a 
flowchart in Fig. 13. 

[0086] In Fig. 13, step 501 through step 505 are the 
same as in Fig. 5, and step 1001 is the same as in Fig. 
10, and hence they are not explained herein. Step 1301 
js for reproducing the files in every detected group, and 
as explained in embodiments 1 to 3 of the invention, the 
CPU 110 controls reading of file from the storage unit 
101, decoding by the decoder 108. and synchronized 
reproduction of image and audio by the synchronization 
control unit 109. 

[0087] Thus, the user can reproduce the image and 
audio exactly as designated by user's own. 
[0088] According to embodiment 4, in the case of au- 
tomatic reproduction of image and audio by combining 
three types of files, that is, image file, audio file, and im- 
age/audio multiplex file, the image and audio can be re- 
produced synchronously by freely and easily combing 
the three types of files by the user, without dividing the 
image and audio multiplexed data into image portion 
and audio portion , in the format maintaining the synchro- 
nized relation of image and audio, or, as required, by 
dividing into image portion and audio portion. 
[0089] In embodiment 4 of the invention, when group- 
ing files, an image/audio multiplex file designated by 
character string M CMP_SRC" and files designated by 
character strings "IMG_SRC" and "SND.SRC" are not 
mixed. That is, when placing a image/audio multiplex 
file designated by character string "CMP_SRC H in one 
group, one group is formed by a imago/audio multiplex 
file designated by character string "CMP_SRC" only. For 
example, in Fig. 12, in the group of which GID is 002, 
only the image/audio multiplex file designated by char- 
acter siring "CMP_SRC" is included. The reason for 
such definition is explained below. 
[0090] Suppose, for example as shown in Fig. 15, to 
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start reproduction of image file B and audio file A syn- 
chronously, reproduce audio file A and then audio file B, 
and reproduce the image and audio of the audio/image 
multiplex file B synchronously, and finally reproduce the 
image file C and audio portion of the image/audio mul- 
tiplex file A synchronously. In this case, as an example 
of creating an automatic reproduction control file, the fol- 
lowing procedure is considered as explained above. 
First, by pressing the new reproduction control file cre- 
ate key 208, editing of automatic reproduction control 
file 105 is started, and after displaying the image file B 
in the display unit 112 by manipulating the image file re- 
production mode select key 201 , send key 204, and re- 
turn key 205, this file name is stored in the CPU 110 in 
succession to the character string "IMG.SRC" by the 
decision key 206. Next, similarly, by manipulating the 
image/audio multiplex file reproduction mode select key 
203, send key 204, and return key 205, the image/audio 
multiplex file B is reproduced, and this file name is stored 
in the CPU 110 in succession to the character string 
M CMP_SRC" by the decision key 206. Again, by manip- 
ulating the image file reproduction mode select key 201 , 
send key 204, and return key 205, the image file C is 
displayed in the display unit 112, and this file name is 
stored in the CPU 110 in succession to the character 
string "IMG.SRC" by the decision key 206. As a result, 
the file names to be reproduced are stored in the tem- 
porary memory of the CPU 1 1 0 in the sequence of image 
file B, image/audio multiplex file B, and image file C. 
[0091] Concerning the audio files, in the desired se- 
quence of reproduction, audio file A, audio file B, and 
audio portion of image/audio multiplex file A are repro- 
duced from the speaker 113 by manipulating the audio 
file reproduction mode select key 202, send key 204, 
and return key 205, and these file names are stored in 
the CPU 110 in succession to the character string 
"SND_SRC M by the decision key 206. By pressing the 
group key 207, the selected files are formed as a group, 
and it is written into the automatic reproduction control 
file together with the group ID number (for example, 
001). 

[0092] The automatic reproduction control file 105 
created in this procedure is shown in Fig. 16. In this au- 
tomatic reproduction control file 1 05, six files are formed 
as a group by one group ID (GID) as shown in Fig. 16. 
[0093] Next, suppose to reproduce plural files auto- 
matically by the automatic reproduction control file 105. 
[0094] In the automatic reproduction control file 105 
in Fig. 1 6, image file B, image/audio multiplex file B, and 
image file C are arranged in this order, and it is known 
that the images are reproduced in the same sequence. 
As for the audio files, audio file A, audio file B, and audio 
portion of image/audio multiplex file A are reproduced 
in this sequence, and it is also known that the audio files 
are reproduced so as to synchronize start of reproduc- 
tion of the image file to be reproduced first among the 
image files included in the group and start of reproduc- 
tion of the audio file to be reproduced first among the 



audio files included in the same group, that is, to syn- 
chronize start of reproduction of image file B and start 
of reproduction of audio file A. However, the timely se- 
quence of the audio portion of image/audio multiplex file 
5 B, audio file B, and audio portion of image/audio multi- 
plex file A cannot be judged from the description of the 
automatic reproduction control file 105 in Fig. 16. Spe- 
cifically, the audio file A to be synchronized with the im- 
age file B to be reproduced before the image/audio mul- 
10 tiplex file B must be reproduced before the audio portion 
of image/audio multiplex file B, but nothing is judged 
from the automatic reproduction control file in Fig. 16 
except that the audio file B and audio portion of image/ 
audio multiplex file A are reproduced after audio file A. 
is [0095] Accordingly, it the automatic reproduction con- 
trol file is created in the same procedure as explained 
so far, the information for specifying the timely sequence 
of the audio portion of image/audio multiplex file B, audio 
file B, and audio portion of image/audio multiplex file A 
20 must be added to the automatic reproduction control file. 
In this case, the description in the automatic reproduc- 
tion file is complicated, and the apparatus for reproduc- 
ing the image and audio automatically on the basis of 
this automatic reproduction control file requires a heavy 
25 burden for analyzing the automatic reproduction control 
file. Specifically, the reproducing apparatus is required 
to have hardware or software for more complicated 
analysis of automatic reproduction control file. It also 
adds to time and labor for creating the automatic repro- 
30 duction control file, which means a heavier burden for 
the user. 

[0096] Accordingly, as shown in Fig. 12 and Fig. 14, 
when designating the image/audio multiplex file by char- 
acter string tt CMP_SRC" to be handled as one file re- 

35 producing image and audio synchronously, without mix- 
ing with the files designated by character strings 
H IMG_SRC" and "SND_SRC H , one group is composed 
of only one image/audio multiplex file designated by 
character string tt CMP_SRC". That is, in the case of the 

40 file in which the synchronized relation of image and au- 
dio is complete within one file, in the nature of the file, 
such as an image/audio multiplex file, by including the 
files having the same format as one group, the timely 
sequence of files is not unclear if the automatic repro- 

45 duction control file is composed together with other file 
not having synchronized relation of Image and audio. 
[0097] As clear from embodiment 3 of the invention, 
it is no problem if a plurality of files designated by 
"CMP_SRC" are placed in one group. 

so [0098] Thus, by defining the file creating method in 
such manner, it is not necessary to add the information 
for specifying the timely sequence of audio file or audio 
portion of image/audio multiplex file to the automatic re- 
production control file, and by a simple description of the 

55 automatic reproduction control f ile, the timely sequence 
of files can be clearly defined. Therefore without adding 
to the load for analysis of automatic reproduction control 
file in the apparatus for reproducing image and audio 



11 



NSDOCID <EP_ 1199892A1_I_> 



21 EP1199 

ausomatically. the user is free to create automatic repro- 
duciion control file without any extra burden. 
[0099] As an example of automatic reproduction con- 
trol file for reproducing image and audio in the synchro- 
nized relation as shown in Fig. 15, a case as shown in 5 
Fig. 17 may be also considered. As shown in Fig. 17, 
the portion showing reproduction of image/audio multi- 
plex tile B is supposed to be GID = 002, and when it is 
separated from the GID designating other file, it is evi- 
dent from the explanation so far that the timely se- 10 
quence of the files can be clearly defined. 
[0100] By mistake, if the user creates an automatic 
reproduction control file including the file designated by 
character string "CMP_SRC", and files designated by 
character strings "IMG_SRC M and M SND_SRC" in the '5 
same group, such wrong operation is indicated by error 
message in the display unit 1127s6~that the user may 
be instructed to create the automatic reproduction con- 
trol file correctly. 

20 

(Embodiment 5) 

[0101] In embodiments 1 to 4 of the invention, by de- 
scribing a group consisting of plural files in the automatic 
reproduction control file 1 05, three types of files of image 25 
file, audio file, and image/audio multiplex file are com- 
bined, and image and audio are produced synchronous- 
ly. A next example relates to reproduction by rearrang- 
ing the group described in the automatic reproduction 
control file 105. 30 
[0102] The configuration of the image and audio re- 
producing apparatus of embodiment 5 is the same as 
shown in the block diagram in Fig. 1 , and the explanation 
is omitted. Fig. 18 is a layout diagram of the operation 
panel of the image and audio reproducing apparatus in 35 
embodiment 5 of the invention. The difference from the 
layout shown in Fig. 2 is that a group reproduction mode 
select key 1 801 and a group edit end key 1 802 are newly 
provided in Fig. 18. The display unit 112 and speaker 
113 are the same as in Fig. 2. 40 
[0103] The keys 201 to 209 are the same as in the 
image and audio reproducing apparatus in Fig. 2, and 
the same explanation is omitted. The principal functions 
of the group reproduction mode select key 1 801 and the 
group edit end key 1802 are as follows. *s 
[01 04] When the group reproduction mode select key 
1 801 is pressed, the operation is set in the mode of re- 
producing the image and audio in each group described 
in the automatic reproduction control file 1 05 on the stor- 
age unit 101 , and the CPU 110 controls the reading unit so 
1 06 according to the description of the automatic repro- 
duction control file 105, and roads out the image file, 
audio file, and image/audio multiplex file recorded in the 
storage unit 101, and reproduces the image and audio 
in the group unit. The group edit end key 1802 is a key 55 
for terminating the rearrangement of the group. 
[0105] In embodiment 5 of the invention having such 
configuration, the operation is explained below by refer- 
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ring also to Fig. 19 showing the content of automatic re- 
production control file. 

[0106] For example, suppose the file shown in Fig. 3. 
and automatic reproduction control file "AUTOPLY4" 
having the content shown in Fig. 12 as used in explana- 
tion of embodiment 4 of the invention are stored in the 
storage unit 101 . In this case, suppose to reproduce by 
inverting the sequence of reproduction of the group of 
group ID number 001 (a group consisting of image file 
B and audio file A) and the group of group ID number 
003 (a group consisting of image file C and audio portion 
of image/audio multiplex file A). In this case, first of all, 
a new automatic reproduction control file is created in 
the following procedure. 

[01 07] To begin with , the user presses the new repro- 
duction control file create key 208 in the same way as 
in the case of embodiment 1 of the invention. As a result, 
a file in a file name of, for example, "AUTOPLY5" is cre- 
ated on the storage unit 101 . 

[0108] Next, the user presses the group reproduction 
mode select key 1801 for selecting the group to be re- 
produced automatically, and the image and audio are 
reproduced in each group described in the automatic re- 
production control file. At this time, for example, if plural 
automatic reproduction control files are stored on the 
storage unit 101 , in embodiment 5 of the invention, the 
automatic reproduction control file to be referred to after 
pressing the group reproduction mode select key 1801 
is supposed to be "AUTOPLY4 M , and it is supposed that 
the grouped image and audio are reproduced according 
to the description of "AUTOPLY4" by pressing of the 
group reproduction mode select key 1801 After press- 
ing of the group reproduction mode select key 1 801 , the 
group consisting of image file B and audio file A is re- 
produced by the display unit 1 12 and speaker 113, and 
the reproduction object is changed by the send key 204 
or return key 205. As a result, the group consisting of 
image and audio portions of image/audio multiplex file 
B, and the group consisting of image file C and audio 
portion of image/audio multiplex file A are reproduced. 
[01 09] Suppose the user wishes to reproduce first the 
image file C and audio portion of image/audio multiplex 
file A, and the image and audio portions of the image/ 
audio multiplex file B, and finally image file B and audio 
file A. In this case, by using the send key 204 or return 
key 205, first the group consisting of image file C and 
audio portion of image/audio multiplex file A is repro- 
duced. Then the decision key 206 is pressed. The CPU 
1 1 0, as the decision key 206 is pressed, stores the group 
ID number of the reproduced group, and character string 
and lile name described in the group in the temporary 
memory. That is, the description from line 6 to line 8 in 
Fig. 1 2 is directly stored in the temporary memory. 
[0110] Next, the second group desired to be repro- 
duced, that is, the group consisting of image and audio 
portions of the image/audio multiplex file B is repro- 
duced in the same procedure, and the decision key 206 
is pressed, and then the group ID number of the repro- 
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riuccri group, and character string and file name de- 
scribed in the group are stored in the temporary mem- 
ory. That is. the description from line 4 to line 5 in Fig. 
1 2 is directly stored in the temporary memory. The final 
group, that is, the group consisting of image file B and 
audio file A is reproduced in the same procedure, and 
ihc decision key 206 is pressed, and then the group ID 
number of the reproduced group, and character string 
,ird lilc name described in the group are stored in the 
icmpDrary memory. That is : the description from line 1 
to line 3 in Fig. 12 is directly stored in the temporary 
m-jnory 

[0111] In this state, when the group edit end key 1 802 
is piessed : the CPU 110 controls the writing unit 107, 
rird wnlcs the content stored in the temporary memory 
oi the CPU 110 into the automatic reproduction control 
die "AUTOPLY5" on the storage unit 101 . At this time, 
the qtoup ID number (GID) is newly assigned from the 
cv.ihci number of line. 

(01 12] For example, the first gioup ID number is 001 , 
the second group ID number is 002 : and the final group 
ID number is 003. Fig. 1 9 shows the content of the writ- 
ten automatic reproduction control file "AUTO PLY 5 M . In 
Fig. 19. GID is the group ID number, and in the example 
in Fig. 19. three groups shown in Fig. 12 are changed 
in the sequence and assigned with new GID numbers. 
[01 1 3] This is the procedure of creating the automatic 
reproduction control file "AUTOPLY5". By using the au- 
tomate reproduction control file "AUTOPLY5" thus cre- 
ated, image and audio reproduction is started in syn- 
chronized state in each group, as shown in embodiment 
4 of the invention. At this time, in the reproduction using 
the automatic reproduction control file "AUTOPLYB'V the 
image and audio can be reproduced in a different se- 
quence from the case of automatic reproduction control 
file "AUTOPLY4" shown in embodiment 4 of the inven- 
tion. The mode of image and audio reproduction by us- 
ing the automatic reproduction control file M AUTOPLY5 M 
is schematically shown in Fig 20. 
[0114] Thus, according to embodiment 5 of the inven- 
tion, when reproducing image and audio automatically 
by combining three types of files, that is. image file, au- 
dio file, and image/audio multiplex file, the groups de- 
scribed in the automatic reproduction control file 1 05 
can be freely changed in order and reproduced. 
[01 1 5] In embodiment 5 of the invention, the automat- 
ic reproduction control file to be i efei red to after pressing 
the group reproduction mode select key 1801 is 
M AUTOPLY4 M , but not limited to this, for example, if there 
are plural automatic reproduction control files on the 
memory 1 01 , the user can arbitrarily select a desired 
automatic reproduction control file to be reproduced au- 
tomatically. In this case, the groups described in plural 
automatic reproduction control files can be rearranged, 
and a new automatic reproduction control file can be 
created. For example, a function may be added for se- 
lecting a plurality of automatic reproduction control files 
to be referred to in reproduction after pressing of the 



group reproduction mode select key 1801 , and in each 
group described in these plural automatic reproduction 
control files, the image and audio are reproduced by us- 
ing the keys in the same procedure as mentioned above, 
5 and a desired file for automatic reproduction is selected 
by pressing the decision key 206, and a new automatic 
reproduction control file (for example, "AUTOPLYS-I") 
may be edited. For example, as shown in Fig. 21, the 
groups described in the automatic reproduction control 
w files M AUTOPLY2" and "AUTOPLY5" explained above 
may be reconstructed with the group consisting of image 
file A and audio file B described in "AUTOPLY2" shown 
in Fig. 7, and the group consisting of image file C and 
audio portion of image/audio multiplex file A described 
is in "AUTOPLY5" shown in Fig. 19, and an automatic re- 
production control file "AUTOPLY5-V is newly edited, 
and the image"a7Td audio are reproduced by using the 
"AUTOPLY5". Thus, as shown in Fig. 21 , by editing two 
automatic reproduction control files., anew automatic re- 
20 production control file can be created. 



(Embodiment 6) 

[0116] The image and audio reproducing apparatus 
25 explained so far can reproduce the image and audio ac- 
cording to the content of the automatic reproduction 
control file, but in the apparatus not having such auto- 
matic reproduction function, as a matter of course, such 
image and audio reproduction cannot be executed. Ac- 
30 cordingly, when the image and audio to be reproduced 
automatically are converted into other versatile format, 
they can be reproduced in other apparatus. In embodi- 
ment 6 of the invention , therefore, it is proposed to newly 
create a data file capable of obtaining the same repro- 
35 duction result as the result of reproduction of image and 
audio according to the description of the automatic re- 
production control file 105. 

[01 17] Fig. 22 is a block diagram of an image and au- 
dio reproducing apparatus in embodiment 6 of the in- 

40 vention. In the diagram, reference numerals 101 to 113 
are the same as in embodiment 1 shown in Fig. 1 , and 
the same explanation is omitted, only the portions dif- 
ferent from Fig. 1 are described below. 
[0118] In Fig. 22, format converting unit 2201 is 

45 means for converting the format of image file 102, audio 
file 103, and image/audio multiplex file 104 being read 
out Irom the storage unit 101. A recording unit 2202 is 
means for recording the file converted in format by the 
format converting unit 2201 , which may be either fixed 

so means such as a hard disk device, or detachable means 
such as flash memory card. 

[01 19] In embodiment 6 of the invention having such 
configuration, the operation is explained. 
[0120] For example, the file shown in Fig. 3, and au- 
55 tomatic reproduction control file "AUTOPLY4" having 
the content as shown in Fig. 12 explained in embodi- 
ment 4 of the invention are stored in the storage unit 
101 . Supposing to convert the image and audio to be 
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reproduced according to the description of automatic re- 
production control file "AUTOPLY4" into other format 
ana store, the CPU 110 reads out the necessary image 
file, audio file, and image/audio multiplex file from the 
storage unit 1 01 , and controls the format converting unit 5 
2201 to convert these files into data of a different format 
composed of image and audio so as to maintain syn- 
chronized relation of image and audio as described in 
the automatic reproduction control file. As the format af- 
ter conversion, for example, since the image and audio 10 
are the object of automatic reproduction, various files 
being readout from the storage unit 101 may be com- 
bined into one format of image/audio multiplex file. As 
a result, the image/audio multiplex file created by the 
format converting unit 2201 can be reproduced in any 15 
apparatus capable of reproducing an image/audio mul- 
tiplex file. 

[0121] Thus, according to embodiment 6 of the inven- 
tion, ihe picture and sound reproduced by the image and 
audio reproducing apparatus explained in embodiments 20 
1 to 5 of the invention can be similarly reproduced in 
other apparatus not having the function of automatic re- 
production according to the automatic reproduction con- 
trol file. 

[0122] In the configuration shown in Fig. 22, various 25 
files being read out from the storage unit 101 are con- 
verted in format by the format converting unit 2201 , but 
as shown in Fig. 23, alternatively, the image signal and 
audio signal decoded by the decoder 1 08 may be coded 
again by an encoder 2301 , and recorded by recording 30 
unit 2302. In this case, too, the picture and sound repro- 
duced by the image and audio reproducing apparatus 
explained in embodiments 1 to 5 of the invention can be 
similarly reproduced in other apparatus not having the 
function of automatic reproduction according to the au- 35 
tomatic reproduction control file. 
[0123] In embodiment 6 of the invention, the file for- 
mat after being converted by the format converting unit 
2201 is not particularly specified, and evidently the in- 
vention is effective and realized by any format. The file *o 
format after conversion described herein is a image/au- 
dio multiplex file having image and audio multiplexed 
and stored in one file format, but it is not limited. For 
example, depending on the image and audio reproduc- 
ing apparatus, independent image file and audio file <*s 
may be regarded as a pair, and picture and sound are 
reproduced simultaneously, and considering reproduc- 
tion in such apparatus, the format converting unit 2201 
may be designed to compile image in one file of image 
data only, compile audio in one file of audio data only, so 
and create a pair of image file and audio file compiling 
audio and imago respectively. Both files relating to im- 
age and audio may not be always created in the format 
converting unit 2201 , and a file recording either picture 
or sound only may be created as required. 55 
[0154] Similarly, the invention is effective regardless 
of the coding method of the encoder 2301 . Further, the 
signal to be coded by the encoder 2301 may be either 
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image signal or audio signal. 

[0125] In embodiment 6 of the invention, the informa- 
tion recorded in the recording unit 2202 and 2302 may 
be either digitized information or analog signal. The re- 
cording unit 2202 and 2302 may be disk device such as 
a hard disk, or semiconductor memory or magnetic tape. 
[0126] Also in embodiment 6 of the invention, the in- 
formation converted in format or coded by the format 
converting unit 2201 or encoder 2301 may be stored in 
the storage unit 101 . In this case, two recording unit are 
not needed in one apparatus, and the apparatus size is 
reduced and the power consumption is saved. 

(Embodiment 7) 

[0127] In embodiments 1 to 4 of the invention, by de- 
scribing the group consisting oTplural files in the auto- 
matic reproduction control file 105, picture and sound 
are reproduced synchronously by combining three 
types of files, that is, image file, audio file, and image/ 
audio multiplex file. Herein, when reproducing image 
and audio according to the description of automatic re- 
production control file 105, a case of reproducing by 
adding different information is explained. 
[0128] Fig. 24 is a block diagram of an image and au- 
dio reproducing apparatus in embodiment 7 of the in- 
vention. The difference of the block diagram in Fig. 24 
from the block diagram in Fig. 1 relating to embodiments 
1 to 4 of the invention is that an additional information 
file 2401 is stored in the storage unit 101 and that a de- 
coding/combining unit 2402 is newly provided, in Fig. 
24. 

[0129] In Fig. 24, reference numerals 101 to 113 are 
the same as in the image and audio reproducing appa- 
ratus shown in Fig. 1 , and the same explanation is omit- 
ted, only the portions different from Fig. 1 are described 
below. 

[0130] The additional information file 2401 is a file 
storing combined information added to image file 102, 
audio file 103, and image/audio multiplex file 104, and 
includes, for example, frame image and title image dis- 
played and overlaid on the image, date of taking image, 
place of taking, file name, camera operator's name, ad- 
dress and telephone number, comments on image and 
other text indicated by character codes, numerical data 
and symbols showing the conditions of taking, and spe- 
cial sound effects reproduced together with the sound. 
The decoding/combining unit 2402 is means for obtain- 
ing a signal combining image file 102, audio file 103, 
image/audio multiplex file 104, and additional informa- 
tion file 2401 . In Fig. 24, for the sake of simplicity, only 
one additional information file 2401 is shown, but plural 
files may be present. 

[01 31 ] Fig. 25 is a block diagram showing the config- 
uration of the decoding/combining unit 2402. In Fig. 25, 
a decoder 2501 is means for decoding the image file, 
audio file, and image/audio multiplex file, in the same 
manner as the decoder 108 shown in Fig. i . A decoder 
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2502 is similar means, and it is provided for decoding 
the additional information file 2401. A combining unit 

2503 is means for overlaying or combining the signals 
decoded by the decoders 2501 and 2502, and issuing 
a combined output. 

[0132] Fig. 26 is a layout diagram of operation panel 
of the image and audio reproducing apparatus in em- 
bodiment 7 of the invention. The difference from the lay- 
out shown in Fig. 2 is that an additional information re- 
production mode select key 2601 is newly provided in 
Fig. 26. 

[0133] The keys 201 to 209 are the same as the keys 
of the image and audio reproducing apparatus shown in 
Fig. 2, and are not explained here. The principal function 
of the additional information reproduction mode select 
key 2601 is as follows. In Fig. 26, the display unit 112 
and speaker 113 are the same as in Fig. 2. 
[0134] When the additional information reproduction 
mode select key 2601 is pressed, the CPU 110 controls 
the reading unit 1 06, and reads out one of the additional 
information files 2401 recorded in the storage unit 101, 
and it is decoded by the decoder 2402, and the content 
of the additional information file is reproduced in the dis- 
play unit 1 1 2 or speaker 1 1 3. At this time, the signal de- 
coded by the decoder 2502 is issued through the com- 
bining unit 2503, but nothing is processed in the com- 
bining unit 2503, and the output of the decoder 2502 is 
directly issued. 

[0135] Fig. 27 is a schematic diagram of additional in- 
formation file stored in the storage unit 101 . In Fig. 27, 
frame image files A. B, and text files A, B are additional 
information files for modifying the image files or adding 
information by characters by combining with image files. 
Sound effect files A, B are additional information files 
which are combined with audio files to superimpose 
special sound effects on the reproduced sound. In the 
following explanation, it is supposed that the six addi- 
tional information files shown in Fig. 27 are recorded on 
the storage unit 101 together with eight files consisting 
of image files, audio files : and image/audio multiplex 
files shown in Fig. 3. 

[0136] In embodiment 7 of the invention having such 
configuration, the operation is explained below by refer- 
ring mainly to the difference from embodiment 1 of the 
invention. 

[0137] When the additional information file 2401 re- 
corded on the storage unil 101 is added to Ihe image 
file 102, audio file 103, and image/audio multiplex file 
1 04, and reproduced, an automatic reproduction control 
file as shown in Fig. 28 is created in the following pro- 
cedure. 

[0138] Using eight files shown in Fig. 3, suppose the 
user wishes to reproduce the picture and sound in the 
same combination as shown in embodiment 1 of the in- 
vention, and further reproduce by combining the image 
file B with the frame image file B, the image portion of 
image/audio multiplex file A with text file A, and audio 
file A with sound effect file B. 



[0139] To begin with, the user presses the new repro- 
duction control file create key 208. By this key input, ed- 
iting of automatic reproduction control file is newly start- 
ed, and the CPU 110 controls the writing unit 107, and 
5 creates a file in a file name of, for example, M AUTOPLY6 H 
on the storage unit 101. 

[0140] Next, the user presses, in the same way as in 
embodiment 1 of the invention, the image file reproduc- 
tion mode select key 201 for selecting the image to be 
10 reproduced automatically, and the image file on the stor- 
age unit 101 is reproduced and displayed in the display 
unit 112, and by using the send key 204 or return key 
205, the image file B to be reproduced first is displayed 
in the display unit 112. Then the decision key 206 is 
15 pressed. By pressing of the decision key 206, the CPU 
1 1 0 records the file name of the image file B in its internal 
temporary memory in succession to the character string 
M IMG„SRC= W . To select the additional information file to 
be combined with the image file B, the additional infor- 
20 mation reproduction mode select key 2601 is pressed, 
and the additional information file 2401 on the storage 
unit 101 is reproduced. At this time, if the additional in- 
formation file to be reproduced is a frame image file, this 
image is displayed on the display unit 112, and in the 
25 case of a text file, the text included in the file is displayed 
on the display unit 112, or in the case of a sound effect 
file, the sound effect is reproduced from the speaker 
113. In the same procedure as when the image file is 
selected, the additional information file to be superim- 
30 posed is reproduced, and the decision key 206 is 
pressed, so that the additional information file name is 
stored in the temporary memory of the CPU 11 0. At this 
time, the file name of the frame image file B is stored, 
in succession to the character string "ADD_SRC° to 
35 show it is an additional information file. 

[0141] in the similar procedure, after selection of im- 
age portion of the image/audio multiplex file A, text file 
A is selected as additional information file, and after se- 
lection of audio file A, sound effect file B is selected as 
40 additional information file, and then the file name of im- 
age/audio multiplex file B is selected as the audio infor- 
mation, and in this way the file names of the image file, 
audio file, and image/audio multiplex file desired to be 
reproduced, and the file names of the additional infor- 
45 mation files are stored in the temporary memory of the 
CPU 110 together with the character string explained 
above. 

[0142] When the group key 207 is pressed in this 
state, the CPU 110 recognizes these selected files, that 

so is, image file B and image portion of image/audio multi- 
plex file A, audio file A and audio portion of image/audio 
multiplex file B, and the corresponding additional infor- 
mation files, that is, frame image file B, text file A, and 
sound effect file B, as one group, and provides the group 

55 with an original ID number (for example, 001 ), and con- 
trols the writing unit 107, and writes the group original 
ID number and the content stored in the temporary 
memory of the CPU 1 1 0, sequentially in the order of stor- 
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ago. in the automatic reproduction control file 
"AUTOPIY6" on the storage unit 101. Fig. 28 shows the 
conicnt of the written automatic reproduction control file 
M AUTOPLY6". 

[0143] Of the files indicated by "IMG.SRC", 5 
"SND_SRC M , the file having the smaller number of lines 
is the file to be reproduced first, and the file designated 
by "ADD_SRC M is the additional information file to be 
ccmbined with the file described in the immediately pre- 
ceding line as counted from the earlier number line. io 
[01 44] This is the procedure of creating the automatic 
rcoroduction control file 105. Reproduction of file ac- 
cording to the automatic reproduction control file 105 
shown in Fig. 28 is explained. 

[0145] When the user presses the automatic repro- '5 
diction key 209, the CPU 110 controls the reading unit 
106 roads out the automatic reproduction control file 
105 and acquires its content. The CPU 110 searches 
lo* the GID from the first line of the automatic reproduc- 
tion conliol file, and recognizes from the first G ID to just 20 
before next GID as one group. The final group in the 
automatic reproduction control file 105 is from the final 
GID to the final tine of the file 

[0146] The automatic reproduction control file in- 
cludes file names in the sequence of reproduction. 25 
Since the file designated by "ADD_SRC" is an additional 
information file, this additional information file is com- 
bined and reproduced when reproducing the file de- 
scribed in the line immediately preceding the description 
of"ADD_SRC". Specifically, the CPU 110 sends the in- .30 
formation showing the file to be combined to the com- 
bining unit 2503 by way of the synchronization control 
unit 1 09. The combining unit 2503 receives signals sent 
from the decoders 2501 and 2502. and judges the one 
to be combined out of them according to the instruction 35 
of the CPU 11 0 through the synchronization control unit 
1 09. and executes the combining process. 
[0147] As a result, when reproducing the image file B, 
the frame image file B is combined, and when reproduc- 
ing the image portion of image/audio multiplex file A, text *o 
file A is combined, and as for the sound, when repro- 
ducing the audio file A. sound effect file B is combined. 
In the audio portion of the image/audio multiplex file B, 
since combination of additional information file is not in- 
structed, the audio portion of the image/audio multiplex *5 
file B is directly reproduced. 

[0148] This series of flow of reproduction of files is 
shown in a flowchart in Fig. 29. In Fig. 29, steps 501 to 
505 are the same as in Fig. 5. and the explanation is 
omitted. Step 2901 is a step for detecting the file name so 
designated by character string "ADD_SRC" showing the 
additional information file in every detected group. At 
step 2902, the CPU 110 reads out the designated files 
and additional information files from the storage unit 
101 . and controls decoding by the decoders 250 1,2502, 55 
and combination of picture or sound and additional in- 
formation file by the combining unit 2503. 
[0149] Such mode of reproduction of picture and 



sound is schematically shown in Fig. 30. That is, the im- 
age is reproduced in the sequence of the image file B 
combined with frame image B f and image portion of im- 
age/audio multiplex file A combined with text file A. and 
in synchronism with the start of reproduction of image 
file B, the audio file A combined with sound effect file is 
reproduced, and in synchronism with the image portion 
of the image/audio multiplex file A, the audio portion of 
the image/audio multiplex file B is reproduced in this or- 
der. Thus, the user can reproduce the picture and sound 
in a format combined with additional information file, ex- 
actly as designated by own. 

[0150] Thus, according to embodiment 7, according 
to the description of the automatic reproduction control 
file 105, when reproducing picture and sound in syn- 
chronism by combining three types of files of image file, 
audio file, and image/audio multiplex file, combination 
of additional information file can be instructed, and the 
picture and sound can be freely reproduced in a format 
combined with the additional information file. 
[0151] In embodiment 7 of the invention, two decod- 
ers 2501 and 2502 shown in Fig. 25 may not be always 
needed, and one decoder may be used commonly. 
[0152] As examples of additional information file in 
embodiment 7 of the invention , frame image file, text file , 
and sound effect file are shown, but the information to 
be added to the picture and sound to be reproduced is 
not limited to these files alone 

[0153] Also in embodiment 7 of the invention, 
M ADD_SRC" is used as character string for designating 
the additional information file, but this is only an example 
and is not limited. 

(Embodiment 8) 

[0154] In embodiments 1 to 4 of the invention, by de- 
scribing a group consisting of plural files in the automatic 
reproduction control file 1 05, three types of files of image 
file, audio file, and image/audio multiplex file are com- 
bined, and picture and sound are reproduced synchro- 
nously. In these examples, the reproduction time of pic- 
ture and sound is not particularly mentioned, and this is 
to explain an example of reproducing by designating the 
reproduction time of picture and sound when reproduc- 
ing the picture and sound according to the description 
of automatic reproduction control file 1 05. 
[0155] Fig. 31 is a layout diagram of operation panel 
of an image and audio reproducing apparatus in embod- 
iment 8 of the invention. As compared with the layout in 
Fig. 2, a reproduction time setting key 3101 and a nu- 
merical input dial 3102 are newly provided in Fig. 31 . 
[0156] Keys 201 to 209 arc the same as the keys of 
the image and audio reproducing apparatus shown in 
Fig. 2. and the explanation is omitted. Principal functions 
of the reproduction time setting key 31 01 and numerical 
input dial 3102 are as follows. 

[0157] When the reproduction time setting key 3101 
is pressed, it results in the mode for setting the repro - 
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duction time of the selected file. An arbitrary numerical 
value is entered by rotating the numerical input dial 
3102. 

[0158] The configuration of the image and audio re- 
producing apparatus in embodiment 8 of the invention 
is the same as the block diagram in Fig. 1 , and its ex- 
planation is omitted. 

[0159] Concerning embodiment 8 of the invention 
having such configuration, the operation is explained 
below mainly referring to the points different from em- 
bodiment 1 of the invention. 

[01 60] When reproducing the image file 1 02, audio file 
1 03, and image/audio multiplex file 104 recorded on the 
storage unit 101 by designating the reproduction time, 
the automatic reproduction control file is created in the 
following procedure. 

[0161] Among the eight files shown in Fig. 3, suppose 
the user desires to reproduce the picture and sound in 
the same combination as in embodiment 1 of the inven- 
tion, and to specify each reproduction time. 
[0162] First, the user presses the new reproduction 
control file create key 208. By this key input, editing of 
automatic reproduction control file shown in Fig. 32 is 
newly started, and the CPU 1 1 0 controls the writing unit 
107, and creates a file in a file name of, for example, 
"AUTO PLY 7" on the storage unit 1 01 . 
[01 63] Next, the user presses the image file reproduc- 
tion mode select key 201 for selecting the picture to be 
reproduced automatically in the same way as in embod- 
iment 1 , and reproduces and displays the image file on 
the storage unit 1 01 in the display unit 112, and by using 
the send key 204 or return key 205, the image file B to 
be reproduced first is displayed in the display unit 112. 
Then the decision key 206 is pressed. By pressing of 
the decision key 206, the CPU 1 1 0 records the file name 
of image file B in succession to the character string 
H IMG_SRC=" in its internal temporary memory. Then, to 
designate the reproduction time, when the reproduction 
time setting key 3101 is pressed, a candidate value of 
reproduction time is displayed in the display unit 1 1 2, for 
example, in the unit of seconds. The candidate value 
may be an initial value preset in the apparatus. When 
the user turns the numerical input dial 31 02, the candi- 
date value of reproduction time changes according to 
the rotation, and when a desired reproduction time ap- 
pears, the numerical input dial is stopped. For example, 
to reproduce the image file B for 10 seconds, the nu- 
merical input dial 3102 is turned until the reproduction 
time displayed in the display unit 112 is 10 seconds. 
Then the decision key 206 is pressed. The reproduction 
time displayed in the display unit 112 is stored in the 
temporary memory of the CPU 110. At this time, the re- 
production time is stored in the unit of seconds in suc- 
cession to the character string "PLYJDUR" showing this 
reproduction time. 

[0164] In the similar procedure, after selecting the im- 
age portion of image/audio multiplex file A, reproduction 
time of 15 seconds is set, and also after selection of au- 



dio file A, reproduction time of 20 seconds is set, and 
finally reproduction time of 5 seconds is set for the audio 
portion of image/audio multiplex file B. 
[0165] After completion of selection of the file to be 

5 reproduced and setting of reproduction time, when the 
group key 207 is pressed, the CPU 110 recognizes the 
two selected image file B and image/audio multiplex file 
A, and audio file A and audio portion of image/audio mul- 
tiplex file B as one group, and provides with an original 

10 group ID number (for example, 001), and controls the 
writing unit 1 07, and writes the group original ID number 
and the content stored in the temporary memory of the 
CPU 110 in the automatic reproduction control file 
"AUTOPLY7" on the storage unit 101 in the sequence 

15 of storage. Fig. 32 shows the content of the written au- 
tomatic reproduction control file "AUTOPLY7". 
[0166] Of the files indicated by "IMG_SRC M and 
M SND_SRC'\ the file of the smaller number of lines is 
the file to be reproduced first in lime, and the numerical 

20 value designated by "PLY_DUR" is the reproduction 
time shown in the unit of seconds. In the example of au- 
tomatic reproduction control file "AUTOPLY7" in Fig. 32, 
the reproduction time of image file B is 10 seconds, the 
reproduction time of image portion of image/audio mul- 

25 tiplex file A is 1 5 seconds, the reproduction time of audio 
file A is 20 seconds, and the reproduction time of audio 
portion of image/audio multiplex file B is 5 seconds. The 
reproduction time designated by "PLY_DUR" is the du- 
ration of reproduction of the file described in the line im- 

30 mediately before as counted from the line of the earlier 
number. 

[0167] This is the procedure of creating the automatic 
reproduction control file 105. Reproduction of files ac- 
cording to the automatic reproduction control file 105 
35 shown in Fig. 32 is explained below. 

[0168] When the user presses the automatic repro- 
duction key 209, the CPU 110 controls the reading unit 
106, reads out the automatic reproduction control file 
105, and acquires its content. The CPU 110 searches 
40 for the GID from the first line of the automatic reproduc- 
tion control file, and recognizes from the first GID to just 
before next GID as one group. The final group in the 
automatic reproduction control file 105 is from the final 
GID to the final line of the file. 
45 [0169] The automatic reproduction control file in- 
cludes file names in the sequence of reproduction. 
Since the numerical value designated by "PLY_DUR U is 
the reproduction time in the unit of seconds, when re- 
producing the file described in the line immediately be- 
so fore description of "PLYJDUR", it is reproduced only for 
this duration. Specifically, the CPU 1 1 0 sends the infor- 
mation showing the files to be reproduced synchronous- 
ly and the information about the reproduction time to the 
synchronization control unit 109. The synchronization 
55 control unit 109 monitors the progress of decoding of 
the picture and sound to be reproduced, and upon com- 
pletion of decoding of the data corresponding to the des- 
ignated reproduction time of picture and sound, decod- 
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ing by the decoder 108 is stopped, and decoding of im- 
age and audio to be decoded next is started. Thus, the 
reproduction time of picture and sound is controlled. 
[0170] As a result, the image file B is reproduced for 
10 seconds, and the image portion of the succeeding 
image/audio multiplex file A is reproduced for 15 sec- 
onds. Synchronously with reproduction of image file B, 
reproduction of audio file A is started, and after repro- 
duction for 20 seconds, the audio portion of the suc- 
ceeding image/audio multiplex file B is reproduced for 
5 seconds, and the series of reproduction of picture and 
sound is terminated. 

[0171] This series of flow of reproduction of files is 
shown in a flowchart in Fig. 33. In Fig. 33, steps 501 to 
505 are the same as in Fig. 5, and the explanation is 
omitted. Step 3301 is a step for detecting the numerical 
value designated by the character string M PLY_DUR" 
showing the reproduction time of each file in each de- 
tected group. At step 3302, the CPU 1 1 0 reads out each 
designated file from the storage unit 1 01 , and instructs 
the files to be reproduced synchronously and the repro- 
duction time to the synchronization control unit 1 09. The 
synchronization control unit 1 09 controls decoding in the 
decoder 1 08, and also controls synchronized reproduc- 
tion of picture and sound and the reproduction time. 
[0172] Such mode of reproduction of image and audio 
is schematically shown in Fig. 34, in which the user can 
reproduce the image and audio exactly for the duration 
designated by own. 

[0173] Thus, according to embodiment 8 of the inven- 
tion, when reproducing the image and audio synchro- 
nously by combining three types of files of image file, 
audio file, and image/audio multiplex file, according to 
the description of the automatic reproduction control file 
105, the reproduction time of each file can be designat- 
ed. Therefore, the image and audio can be freely repro- 
duced for the designated duration. 
[0174] In embodiment 8 of the invention, the repro- 
duction time is designated as the reproduction time of 
each file, but not limited to this, for example, supposing 
the reproduction start time of the first file to be repro- 
duced to be 0, the reproduction time of each file can be 
set by the absolute time from this reproduction start 
time. As shown in Fig. 35, for example, the total of re- 
production time of all files reproduced automatically is 
designated by "PLY_TTL", and the reproduction start 
time of each file is designated by "PLY_TIM", and there- 
fore, similarly, the reproduction time of each file to be 
reproduced can be set. In ; the example in Fig. 35, the 
total lime of automatic reproduction is 25 seconds, the 
reproduction start time of the image file B to be repro- 
duced in the first place is designated at 0 second by the 
character string M PLY_TIM", and the reproduction start 
time of the image portion of the image/audic multiplex 
file A to be reproduced next is designated at 1 0 seconds 
by the character string "PLY_TIJvT, so that the files are 
reproduced in same manner as explained in Fig. 32 and 
Fig. 34. 



[0175] Also in embodiment 8 of the invention, the can- 
didate value of reproduction time displayed in the dis- 
play unit 112 when the reproduction time setting key 
31 01 is pressed is the initial value preset in the appara- 

5 tus, but not limited to this, for example, the time neces- 
sary for reproducing all data stored in the files of image 
file, audio file, and image/audio multiplex file (for exam- 
ple, 5 seconds in the case of image file storing a moving 
image for 5 seconds, and 1 0 seconds in the case of au- 

10 dio file storing audio data for 1 0 seconds) is judged from 
the data in each file, and this value may be first shown 
in the display unit 112. 

[0176] Also in embodiment 8 of the invention, the re- 
production time is set in the unit of seconds, but not lim- 
'5 ited to this, any time unit including fractions is effective 
in the invention. 

[0177] Also in embodiment 8 of the invention, the im- 
age file and image portion of image/audio multiplex file 
may be either a moving image consisting of plural 

20 frames, or a still image consisting of one frame only, and 
if the image is a still image, the still image may be dis- 
played for a designated duration. 
[0178] Also in embodiment 8 of the invention, if the 
image file or image portion of image/audio multiplex file 

25 is a moving image, when reproducing the file for the des- 
ignated time, both a method of starting reproduction 
from the beginning of the moving image and a method 
of setting the reproduction start point freely by the user 
may be considered. When the user designates the re- 

30 production start point arbitrarily, for example, the repro- 
duction start point of the moving image is indicated by 
the time from the beginning portion of the moving image, 
and it is expressed by character string "PLY_STR". and 
it may be described in succession to the character string 

35 "PLYJDUR" in the automatic reproduction control file. 
For example, referring to the automatic reproduction 
control file shown in Fig. 32, assuming the image file B 
to be a moving image of 30 seconds, when reproducing 
the image file B for 1 0 seconds from the point of 3 sec- 

^o onds after the beginning, the automatic reproduction 
control file is described as shown in Fig. 36. The char- 
acter string. "PLY_STR" shows the reproduction start 
point of the moving image, and by designating "3" here- 
in, the reproduction start point of the image file B can be 

^5 designated. In this character string ,, PLY_STR", not lim- 
ited to the image file, the reproduction start point of the 
image portion of image/audio multiplex file or audio file 
may be designated. If the reproduction start point is not 
designated by the character string "PLY_STR", it may 

so be designed to start reproduction of the moving image 
from the beginning unconditionally. 
[0179] Also in embodiment 8 of the invention, in the 
case the reproduction object file is a moving image, if 
the designated reproduction time is longer than the time 

55 necessary for reproducing ail image data stored in the 
fiies of image file and image/audio multiplex file, after 
completion of reproduction of the moving image within 
the designated time, blue background or other image 
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prepared in the apparatus may be displayed. For exam- 
ple, if reproduction of 10 seconds is designated for a 
moving image of 3 seconds, the image is displayed for 
the first 3 seconds, and a blue background is displayed 
for the remaining 7 seconds. As for the audio, similarly, 
if the designated reproduction time is longer than the 
time necessary for reproducing all audio data stored in 
the files of audio file and image/audio multiplex file, after 
completion of reproduction of the audio within the des- 
ignated time, other audio prepared in the apparatus or 
silence may be reproduced. 

[0180] Also in embodiment 8 of the invention, in the 
case the reproduction object file is a moving image, if 
the designated reproduction time is longer than the time 
necessary for reproducing all image data stored in the 
files of image file and image/audio multiplex file, an error 
message may be displayea r on the display unit 112, urg- 
ing the user to set the time correctly for reproducing all 
image data. 

[01 81 ] Also in embodiment 8 of the invention, as char- 
acter strings designating the reproduction time and oth- 
ers, "PLYJXJR", "PLY_TTL y , "PLY_TIM M , and 
M PLY_STR" are used, but these are only examples and 
are not limited. 

[01 82] In all embodiments of the invention, the coding 
method and file format of image file, audio file, and im- 
age/audio multiplex file are not particularly specified: but 
it is evident that the invention is effective regardless of 
coding method orformat of the files. Likely, the invention 
is valid whether the image file, audio file, and image/ 
audio multiplex file are compressed files or non-com- 
pressed files, or whether the files are provided with 
header or additional information or not. 
[0183] Also in all embodiments of the invention, as 
character strings for designating files, "IMG_SRC", 
"SND_SRC\ "CMP_SRC" are used, but these are only 
examples and are not limited. 

[0184] Also in all embodiments of the invention, the 
decoder 1 08 includes a digital-to-analog converter for 
converting a digital signal into an analog signal. 
[0185] Also in all embodiments of the invention, the 
key input of the operation key 11 1 is effected by pressing 
any one of the keys 201 to 208 shown in Fig. 2, but not 
limited to this, for example, necessary operation may be 
selected from a touch panel or menu display on a liquid 
crystal display device. 

[0186] Also in all embodiments of the invention, only 
one automatic reproduction file used in reproduction is 
shown on the storage unit 101 , but plural automatic re- 
production control files may be present on the storage 
unit 101, and the user may freely select one automatic 
reproduction control file used in automatic reproduction. 
In this case, among other examples, all automatic re- 
production control files on the storage unit 101 are dis- 
played on the display unit 112, and the user selects a 
desired automatic reproduction control file by key input 
or the like. 

[0187] Also in all embodiments of the invention, the 



automatic reproduction control file is recorded on the 
storage unit 101 , but not limited to this, for example, it 
may be recorded on other storage unit than the storage 
unit 101. 

5 [0188] Also in all embodiments of the invention, when 
creating the automatic reproduction control file, the pic- 
ture or sound is reproduced, but not limited to this, for 
example, without reproducing the picture or sound, their 
file names may be displayed in the display unit 112, and 

10 a file for automatic reproduction may be selected from 
the file names. 

[01 89] Also in all embodiments of the invention , when 
creating an automatic reproduction control file, the pic- 
ture or sound is reproduced, but as for the picture, a 
15 thumbnail image may be displayed. As forthe audio, on- 
ly a part, for example, the beginning portion may be re- 
produced. 

[0190] Also in all embodiments of the invention, the 
number of frames in Ihe image file and image porlion of 

20 image/audio multiplex file is not particularly specified, 
and the image may be either a still picture of one frame 
or a moving picture consisting of plural frames. 
[0191] Also in all embodiments of the invention, when 
creating an automatic reproduction control file, the pic- 

25 ture or sound is reproduced, but in the case of a moving 
picture consisting of plural frames, only a part of the pic- 
ture may be displayed. 

[0192] Also in all embodiments of the invention, at 
least processing by the CPU 110 can be realized by the 
30 software, and the software may be preliminarily stored 
in the memory or provided from other recording medium . 

INDUSTRIAL APPLICABILITY 

35 [0193] As described herein, according to the inven- 
tion, files of different formats, such as a data file record- 
ing image signals, a data file recording audio signals, 
and a data file multiplexing and recording image signals 
and audio signals, can be freely and easily combined by 

40 the user, or various additional information may be given 
to the picture or sound, and the files may be reproduced 
continuously by changing the sequence of reproduction 
or making other modifications, and the apparatus can 
execute start of reproduction of specified picture and 

45 sound synchronously. 



Claims 

so 1 . A image and audio reproducing apparatus compris- 
ing: 

storage means for storing 
at least one of recorded image signal data, re- 
55 corded audio signal data and recorded image/ 

audio signal multiplex data, and 
reproduction information data describing infor- 
mation about reproduction of said at least one 
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of the recorded image signal data, the recorded 
audio signal data and the recorded image/au- 
dio signal multiplex data or an combination 
thereof, independently and individually. 

5 

2. The image and audio reproducing apparatus of 
claim 1 , wherein the reproduction information data 
includes information for designating data to be re- 
produced among the data stored in the storage 
means. 10 

3. The image and audio reproducing apparatus of 
claim 1 , wherein the reproduction information data 
includes information showing sequence of repro- 
ducing the data stored in the storage means. '5 

4. The image and audio reproducing~apparatus of 
claim 1 , wherein the reproduction information data 
includes information showing duration of reproduc- 
ing the data stored in the storage means. 20 

5. The image and audio reproducing apparatus of 
claim 1 , wherein the reproduction information data 
includes information showing time of reproducing 

the data stored in the storage means. 25 

6. The image and audio reproducing apparatus of 
claim 1 or 2, wherein in the reproduction information 
data, when designating the recorded image/audio 
signal multiplex data, a image signal portion and an 30 
audio signal portion of the recorded image/audio 
signal multiplex data can be designated separately 

as an object of reproduction. 

7. The image and audio reproducing apparatus of 35 
claim 1 or 2, wherein in the reproduction information 
data, the information about reproduction of the data 

is recorded in at least one group, and the one group 
includes at least one piece of information for desig- 
nating data to be reproduced. *o 

8. The image and audio reproducing apparatus of 
claim 1 or 2, wherein in the reproduction information 
data, the information about reproduction of the data 

is recorded in at least one group, and the one group 4 $ 
includes one piece each of information for designat- 
ing one of the recorded image signal data and an 
image signal portion of the recorded image/audio 
signal multiplex data, and information for designat- 
ing one of the recorded audio signal data, and an so 
audio signal portion of the recorded image/audio 
signal multiplex data. 

9. The image and audio reproducing apparatus of any 
one of claims 1, 2, and 7, wherein in the group of 55 
reproduction information data, when writing the in- 
formation for designating the recorded image/audio 
signal multiplex data, the information for designat- 
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ing the recorded image signal data, an image signal 
portion of the recorded image/audio signal multiplex 
data, the recorded audio signal data, or an audio 
signal portion of the recorded image/audio signal 
multiplex data is not written in one group. 

1 0. The image and audio reproducing apparatus of any 
one of claims 1 to 9. wherein new reproduction in- 
formation data is created by editing the reproduc- 
tion information data in a group. 

1 1 . The image and audio reproducing apparatus of any 
one of claims 1 to 1 0, further comprising format con- 
verting means for converting data format of the im- 
age recorded signal data, the recorded audio data, 
or the recorded image/audio signal multiplex data, 
reproduced according to the reproduction informa- 
tion data. 

12. The image and audio reproducing apparatus of any 
one of claims 1 to 11 , further comprising encoding 
means for encoding the image signal and the audio 
signal reproduced according to the reproduction in- 
formation data. 

13. The image and audio reproducing apparatus of any 
one of claims 1 to 12, wherein combined information 
data including information combined with said at 
least one of recorded image signal data, recorded 
audio signal data and recorded image/audio signal 
multiplex data is stored in the storage means, and 
the reproduction information data includes informa- 
tion for designating data to be reproduced, and in- 
formation for designating the combined information 
data. 

14. The image and audio reproducing apparatus of any 
one of claims 1, 2, and 6 to 13, wherein the infor- 
mation for designating the data to be reproduced is 
a name given to the data to be reproduced. 

1 5. The image and audio reproducing apparatus of any 
one of claims 1 to 14, further comprising reproduc- 
ing means for reproducing the image signal and the 
audio signal from data stored in the storage means 
according to the reproduction information data. 

16. The image and audio reproducing apparatus of 
claim 1 5, wherein said reproducing mean is at least 
one of a decoder for combining data, means for 
reading out data from the storage means, and a de- 
coder for decoding data. 

17. The image and audio reproducing apparatus of 
claim 13, wherein the combined information file is 
at least one of an image, an audio and a text con- 
tained in a data file recording the image signal, a 
data file recording the audio signal, and a data file 



EP 1 199 892 A1 



MSDCCID <EF _ 1 199652* ! J_> 



39 



EP 1 199 892 A1 



40 



multiplex recording a image signal or an audio sig- 
nal. 

1 8. A image and audio reproducing method comprising 

the step of 

storing, in at least one storage means, 
at least one of recorded image signal data, re- 
corded audio signal data and recorded image/audio 
signal multiplex data, and 

reproduction information data describing the 
information about reproduction of said at least one 
of the recorded image signal data, the recorded au- 
dio signal data and the recorded image/audio signal 
data multiplex or an combination thereof, independ- 
ently and individually. 

19. The image and audio reproducing method of claim 
18, wherein the reproduction information data in- 
cludes the information for designating the data lo 
be reproduced among the data stored in the storage 
means. 

20. The image and audio reproducing method of claim 
18, wherein the reproduction information data in- 
cludes information showing the sequence of repro- 
ducing the data stored in the storage means. 

21. The image and audio reproducing method of claim 
■\&, wherein the reproduction information data in- 
cludes information showing the duration of repro- 
ducing the data stored in the storage means. 

22. The image and audio reproducing method of claim 
18, wherein the reproduction information data in- 
cludes information showing the time of reproducing 
the data stored in the storage means. 

23. The image and audio reproducing method of claim 
18 or 19, wherein in the reproduction information 
data, when designating the recorded image/audio 
signal multiplex data, a image signal portion and an 
audio signal portion of the recorded image/audio 
signal multiplex data can be designated separately 
as an object of reproduction. 

24. The image and audio reproducing method of claim 
18 or 19, wherein in the reproduction information 
data, the information about reproduction of the data 
is recorded in at least one group, and the one group 
includes at least one piece of information for desig- 
nating data to be reproduced. 

25. The image and audio reproducing method of claim 
18 or 19, wherein in the reproduction information 
data, the information about reproduction of the data 
is recorded in at least one group, and the one group 
includes one piece each of information for designat- 
ing one of the recorded image signal data and an 
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image signal portion of the recorded image/audio 
signal multiplex data, and information for designat- 
ing one of the recorded audio signal data, and an 
audio signal portion of the recorded image/audio 
signal multiplex data. 

26. The image and audio reproducing method of any 
one of claims 18, 19, and 24, wherein in the group 
of reproduction information data, when writing the 
information lor designating the recorded image/au- 
dio signal multiplex data : the information for desig- 
nating the recorded image signal data, an image 
signal portion of the recorded image/audio signal 
data , the recorded audio signal data : and an audio 
signal portion of the recorded image/audio signal 
multiplex data is not written in an identical group. 

27. The image and audio reproducing method of any 
one ol claims 18 to 26, wherein new reproduction 
information data is created by editing the reproduc- 
tion information data in each group. 



28. The image and audio reproducing method of any 
one of claims 18 to 27, wherein data format of the 

25 image recorded signal data, the recorded audio da- 
ta, or the recorded image/audio signal multiplex da- 
ta, reproduced according to the reproduction infor- 
mation data, is converted by format converting 
means. 

30 

29. The image and audio reproducing method of any 
one of claims 18 to 28, wherein the image signal 
and the audio signal reproduced according to the 
reproduction information data are coded by encod- 

35 ing means. 

30. The image and audio reproducing method of any 
one of claims 18 to 29, wherein combined informa- 
tion data including information combined with said 

40 at least one of recorded image signal data , recorded 
audio signal data and recorded image/audio signal 
multiplex data is stored in the storage means, and 
the reproduction information data includes informa- 
tion for designating data to be reproduced, and in- 

45 formation for designating the combined information 
data. 

31 . The image and audio reproducing method of any 
one of claims 18, 19, and 23 to 30, wherein the in- 

so formation for designating the data to be reproduced 
is a name given to the data to be reproduced. 

32. The image and audio reproducing method of any 
one of claims 18 to 31, wherein the image signal 

55 and the audio signal from the data stored in the stor- 
age means according to the reproduction informa- 
tion data are reproduced by using reproducing 
means. 
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33. The image and audio reproducing method of claim 
32. wherein said reproducing mean is at least one 
of a decoder for combining data, means for reading 
out data from the storage means, and a decoder for 
decoding data. 

34. The image and audio reproducing method of claim 
30, wherein the combined information file is at least 
one of an imaae, an audio and a text contained in 
a data file recording image signal, a data file record- 
ing the audio signaL and a data file multiplex record- 
ing a image signal or an audio signal. 
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